2023-10-06T11:28:03.558 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T11:28:03.558 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417 --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} --verbose -- /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/orig.config.yaml 2023-10-06T11:28:03.604 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:28:03.694 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417 branch: wip-yuri3-testing-2023-10-05-0943-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} email: yweinste@redhat.com first_in_suite: false job_id: '7412417' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri3-testing-2023-10-05-0943-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri3-testing-2023-10-05-0943-quincy sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4002 sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri3-testing-2023-10-05-0943-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBETzpEEBd5S9AGT46sVfXh0CmuWgCfE2KYq3uvkh21YMtkyX3fg8MjFIIC2e1shdrgADEeMosd/3XxXSmLp/6sY= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOHDdXoRoHZfnCPeoRS4nfiBlTkz0J5rsDHjszoMelrS3lw0SWeqNkRoWAXZuJv2SwvFbAftTTP/wEmA2FuoN0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - check-counter: counters: mgr: - min: 4 name: finisher-volumes.complete_latency.avgcount - expected_val: 0 name: finisher-volumes.queue_len - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestPerModuleFinsherThread teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:54:57 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T11:28:03.695 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa; will attempt to use it 2023-10-06T11:28:03.695 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa/tasks 2023-10-06T11:28:03.696 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T11:28:03.697 INFO:teuthology.task.internal:Checking packages... 2023-10-06T11:28:03.719 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564' 2023-10-06T11:28:03.719 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T11:28:03.719 INFO:teuthology.packaging:ref: None 2023-10-06T11:28:03.719 INFO:teuthology.packaging:tag: None 2023-10-06T11:28:03.719 INFO:teuthology.packaging:branch: wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:28:03.719 INFO:teuthology.packaging:sha1: 68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:28:03.719 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri3-testing-2023-10-05-0943-quincy 2023-10-06T11:28:03.861 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1274.g68e53a4a 2023-10-06T11:28:03.861 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T11:28:03.866 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T11:28:03.866 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T11:28:03.871 INFO:teuthology.task.internal:Saving configuration 2023-10-06T11:28:03.888 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T11:28:03.893 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:28:03.919 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:22:49.413416', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBETzpEEBd5S9AGT46sVfXh0CmuWgCfE2KYq3uvkh21YMtkyX3fg8MjFIIC2e1shdrgADEeMosd/3XxXSmLp/6sY='} 2023-10-06T11:28:03.941 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:22:49.414813', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOHDdXoRoHZfnCPeoRS4nfiBlTkz0J5rsDHjszoMelrS3lw0SWeqNkRoWAXZuJv2SwvFbAftTTP/wEmA2FuoN0='} 2023-10-06T11:28:03.941 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T11:28:03.946 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T11:28:03.947 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T11:28:03.947 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T11:28:04.011 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efefad5ab80>, signals=[15]) 2023-10-06T11:28:04.012 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T11:28:04.017 INFO:teuthology.task.internal:Opening connections... 2023-10-06T11:28:04.018 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:28:04.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:28:04.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:28:04.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:28:04.155 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T11:28:04.160 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-10-06T11:28:04.179 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-10-06T11:28:04.179 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-10-06T11:28:04.236 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T11:28:04.236 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T11:28:04.236 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-10-06T11:28:04.236 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.4" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T11:28:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T11:28:04.238 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-10-06T11:28:04.267 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-10-06T11:28:04.284 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-10-06T11:28:04.284 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-10-06T11:28:04.340 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.4" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-10-06T11:28:04.341 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T11:28:04.342 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T11:28:04.342 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-10-06T11:28:04.367 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T11:28:04.376 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T11:28:04.381 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T11:28:04.381 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:28:04.383 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2023-10-06T11:28:04.399 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T11:28:04.404 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T11:28:04.404 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:28:04.440 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2023-10-06T11:28:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T11:28:04.456 INFO:teuthology.orchestra.run.smithi177.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T11:28:04.457 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T11:28:04.510 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T11:28:04.523 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T11:28:04.524 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T11:28:04.524 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:28:04.524 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T11:28:04.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:28:04.540 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-10-06T11:28:04.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:28:04.541 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-10-06T11:28:04.596 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T11:28:04.596 INFO:teuthology.task.kernel:Running kernel on smithi177: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T11:28:04.596 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2023-10-06T11:28:04.598 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T11:28:04.598 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-305.25.1.el8_4.x86_64 2023-10-06T11:28:04.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-10-06T11:28:05.994 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:05.994 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-10-06T11:28:06.031 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:06.032 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-10-06T11:28:07.394 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 35 kB/s | 3.5 kB 00:00 2023-10-06T11:28:07.472 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 36 kB/s | 3.5 kB 00:00 2023-10-06T11:28:08.033 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 26 MB/s | 16 MB 00:00 2023-10-06T11:28:08.462 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T11:28:11.454 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-10-06T11:28:12.018 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-06T11:28:12.222 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T11:28:12.222 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T11:28:12.222 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T11:28:12.361 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-10-06T11:28:12.362 INFO:teuthology.orchestra.run.smithi177.stdout:Nothing to do. 2023-10-06T11:28:12.362 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-10-06T11:28:12.468 DEBUG:teuthology.orchestra.run.smithi177:> echo no | sudo yum reinstall kernel || true 2023-10-06T11:28:12.798 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-06T11:28:12.798 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-06T11:28:12.798 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-06T11:28:12.811 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:12.811 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-10-06T11:28:12.960 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-10-06T11:28:12.961 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-10-06T11:28:12.961 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-10-06T11:28:13.076 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2023-10-06T11:28:13.102 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 11:28:11 AM UTC. 2023-10-06T11:28:13.235 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T11:28:13.236 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:13.236 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:13.260 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-10-06T11:28:13.309 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum reinstall -y kernel || true 2023-10-06T11:28:13.441 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:13.441 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-10-06T11:28:13.652 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:13.653 INFO:teuthology.orchestra.run.smithi177.stdout:Unable to read consumer identity 2023-10-06T11:28:13.747 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:01 ago on Fri 06 Oct 2023 11:28:12 AM UTC. 2023-10-06T11:28:13.901 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T11:28:13.901 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:13.901 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-10-06T11:28:13.936 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 11:28:11 AM UTC. 2023-10-06T11:28:13.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2023-10-06T11:28:14.076 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T11:28:14.076 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:14.077 INFO:teuthology.orchestra.run.smithi177.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:14.089 INFO:teuthology.orchestra.run.smithi177.stderr:Error: No packages marked for reinstall. 2023-10-06T11:28:14.146 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T11:28:14.270 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.271 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.271 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.271 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T11:28:14.271 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T11:28:14.271 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T11:28:14.271 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T11:28:14.271 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T11:28:14.349 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-10-06T11:28:14.350 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-10-06T11:28:14.397 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.397 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.398 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2023-10-06T11:28:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2023-10-06T11:28:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2023-10-06T11:28:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.02 2023-10-06T11:28:14.424 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 99.el8_4.1 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Group : System Environment/Base 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 9264631 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Relocations : (not relocatable) 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : Red Hat, Inc. 2023-10-06T11:28:14.425 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : Red Hat, Inc. 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:28:14.426 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T11:28:14.427 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T11:28:14.427 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.427 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T11:28:14.641 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Fri 06 Oct 2023 11:28:12 AM UTC. 2023-10-06T11:28:14.792 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-06T11:28:14.792 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:14.792 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-06T11:28:14.811 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-10-06T11:28:14.867 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T11:28:14.997 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.998 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.998 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:14.998 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T11:28:14.998 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T11:28:14.998 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T11:28:14.998 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T11:28:14.998 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T11:28:15.128 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:15.129 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:15.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2023-10-06T11:28:15.157 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2023-10-06T11:28:15.157 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2023-10-06T11:28:15.157 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.02 2023-10-06T11:28:15.157 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 99.el8_4.1 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Group : System Environment/Base 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 9264631 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Relocations : (not relocatable) 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : Red Hat, Inc. 2023-10-06T11:28:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : Red Hat, Inc. 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:28:15.159 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T11:28:15.160 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T11:28:15.160 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:15.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T11:28:15.820 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2023-10-06T11:28:16.466 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2023-10-06T11:28:18.029 INFO:teuthology.orchestra.run.smithi177.stderr:done 2023-10-06T11:28:18.030 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T11:28:18.044 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.9Nzwejl03w 2023-10-06T11:28:18.044 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9Nzwejl03w 2023-10-06T11:28:18.134 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.9Nzwejl03w 2023-10-06T11:28:18.243 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.9Nzwejl03w is 6KB 2023-10-06T11:28:18.291 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.9Nzwejl03w 2023-10-06T11:28:18.304 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T11:28:18.372 INFO:teuthology.orchestra.run.smithi177.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T11:28:18.372 INFO:teuthology.orchestra.run.smithi177.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T11:28:18.372 INFO:teuthology.orchestra.run.smithi177.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T11:28:18.373 INFO:teuthology.orchestra.run.smithi177.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T11:28:18.374 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:18.543 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2023-10-06T11:28:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:done 2023-10-06T11:28:18.764 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-10-06T11:28:18.782 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.d1BcXeSiiv 2023-10-06T11:28:18.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.d1BcXeSiiv 2023-10-06T11:28:18.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.d1BcXeSiiv 2023-10-06T11:28:19.014 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.d1BcXeSiiv is 6KB 2023-10-06T11:28:19.063 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.d1BcXeSiiv 2023-10-06T11:28:19.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T11:28:19.139 INFO:teuthology.orchestra.run.smithi016.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T11:28:19.140 INFO:teuthology.orchestra.run.smithi016.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T11:28:19.140 INFO:teuthology.orchestra.run.smithi016.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T11:28:19.140 INFO:teuthology.orchestra.run.smithi016.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T11:28:19.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-06T11:28:19.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2023-10-06T11:28:48.575 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T11:28:48.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:28:48.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:28:49.314 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T11:28:49.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:28:49.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:29:06.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T11:29:07.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-10-06T11:29:15.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:29:15.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:29:16.708 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:29:16.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:29:19.006 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T11:29:31.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:29:31.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:29:49.434 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T11:30:04.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:30:04.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:30:04.772 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-10-06T11:30:05.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:30:05.046 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T11:30:05.047 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-10-06T11:30:05.060 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T11:30:05.060 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:30:05.061 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:30:05.061 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T11:30:06.062 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T11:30:06.062 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T11:30:06.139 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2023-10-06T11:30:06.153 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:16.711 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T11:30:28.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:30:28.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:30:28.970 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-10-06T11:30:29.240 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:30:29.240 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-06T11:30:29.240 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-10-06T11:30:29.256 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-06T11:30:29.256 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-06T11:30:29.257 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T11:30:29.257 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T11:30:30.257 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T11:30:30.257 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T11:30:30.330 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-10-06T11:30:30.343 DEBUG:teuthology.parallel:result is None 2023-10-06T11:30:30.343 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T11:30:30.351 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T11:30:30.352 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:30:30.355 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T11:30:30.375 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T11:30:30.385 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T11:30:30.402 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T11:30:30.402 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:30:30.413 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T11:30:30.449 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T11:30:30.457 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T11:30:30.457 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:30:30.481 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T11:30:30.519 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:30:30.521 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:30:30.533 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:30:30.533 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T11:30:30.535 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T11:30:30.542 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T11:30:30.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:30:30.578 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T11:30:30.605 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T11:30:30.613 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T11:30:30.613 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:30:30.647 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T11:30:30.663 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:30:30.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:30:30.790 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:30:30.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:30:30.916 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:30:30.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:30:30.984 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:30:31.028 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T11:30:31.070 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:30:31.150 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T11:30:31.194 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:30:31.194 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T11:30:31.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-10-06T11:30:31.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2023-10-06T11:30:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T11:30:31.359 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T11:30:31.728 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T11:30:31.741 INFO:teuthology.task.internal:Starting timer... 2023-10-06T11:30:31.741 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T11:30:31.762 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T11:30:31.791 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T11:30:31.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-10-06T11:30:31.986 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: [ OK ] 2023-10-06T11:30:31.992 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2023-10-06T11:30:32.057 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: [ OK ] 2023-10-06T11:30:32.058 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T11:30:32.059 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-10-06T11:30:32.088 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-10-06T11:30:32.088 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2023-10-06T11:30:32.115 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2023-10-06T11:30:32.116 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2023-10-06T11:30:32.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T11:30:32.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:30:32.170 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T11:30:32.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:30:32.202 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T11:30:32.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-10-06T11:30:32.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2023-10-06T11:30:32.298 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T11:30:32.475 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T11:30:32.480 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T11:30:32.632 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T11:30:32.649 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T11:30:32.651 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T11:38:51.425 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2023-10-06T11:38:51.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:38:51.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:38:51.497 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-10-06T11:38:51.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-10-06T11:38:51.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:38:51.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T11:38:51.629 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-10-06T11:38:51.693 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T11:38:51.693 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T11:38:51.704 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T11:38:51.704 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:38:51.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:38:51.707 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T11:38:51.707 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:38:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T11:38:51.749 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T11:38:51.762 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T11:38:51.776 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T11:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-10-06T11:38:51.794 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-10-06T11:38:51.808 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T11:38:51.812 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2023-10-06T11:38:51.822 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2023-10-06T11:38:51.823 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T11:38:51.836 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T11:38:51.850 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T11:38:51.888 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:210 Number of sources = 4 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.891 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.919 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:210 Number of sources = 4 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.922 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:38:51.923 INFO:teuthology.run_tasks:Running task install... 2023-10-06T11:38:51.930 DEBUG:teuthology.task.install:project ceph 2023-10-06T11:38:51.930 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}} 2023-10-06T11:38:51.931 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'} 2023-10-06T11:38:51.931 INFO:teuthology.task.install:Using flavor: default 2023-10-06T11:38:51.936 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T11:38:51.937 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T11:38:51.937 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T11:38:51.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:38:51.939 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'tag': None, 'wait_for_package': False} 2023-10-06T11:38:51.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:38:52.080 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T11:38:52.080 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T11:38:52.152 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/ 2023-10-06T11:38:52.153 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1274.g68e53a4a 2023-10-06T11:38:52.188 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T11:38:52.189 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:38:52.189 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T11:38:52.214 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T11:38:52.214 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T11:38:52.258 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2023-10-05-0943-quincy/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T11:38:52.258 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:38:52.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T11:38:52.279 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T11:38:52.285 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T11:38:52.285 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T11:38:52.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T11:38:52.353 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T11:38:52.416 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2023-10-06T11:38:52.418 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2023-10-06T11:38:52.427 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T11:38:52.491 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-10-06T11:38:52.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-10-06T11:38:52.809 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-10-06T11:38:52.906 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-10-06T11:38:53.159 INFO:teuthology.orchestra.run.smithi177.stdout:44 files removed 2023-10-06T11:38:53.210 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T11:38:53.261 INFO:teuthology.orchestra.run.smithi016.stdout:44 files removed 2023-10-06T11:38:53.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T11:38:53.561 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-10-06T11:38:53.684 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-10-06T11:38:54.422 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 209 kB/s | 78 kB 00:00 2023-10-06T11:38:54.489 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 279 kB/s | 78 kB 00:00 2023-10-06T11:38:54.705 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-06T11:38:54.783 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 37 kB/s | 9.2 kB 00:00 2023-10-06T11:38:55.124 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 4.9 kB/s | 1.4 kB 00:00 2023-10-06T11:38:55.148 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 3.5 kB/s | 1.4 kB 00:00 2023-10-06T11:38:55.223 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2023-10-06T11:38:55.255 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2023-10-06T11:38:56.383 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-06T11:38:56.812 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-06T11:38:59.715 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 4.9 MB/s | 24 kB 00:00 2023-10-06T11:39:00.067 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 122 MB/s | 33 MB 00:00 2023-10-06T11:39:00.252 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-10-06T11:39:00.667 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 100 MB/s | 33 MB 00:00 2023-10-06T11:39:05.222 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 128 MB/s | 38 MB 00:00 2023-10-06T11:39:06.112 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 106 MB/s | 38 MB 00:00 2023-10-06T11:39:13.705 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T11:39:13.709 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T11:39:13.716 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T11:39:13.717 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T11:39:13.718 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T11:39:13.719 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T11:39:13.720 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T11:39:13.723 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T11:39:13.724 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T11:39:14.209 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-10-06T11:39:14.248 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================================================== 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================================================== 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T11:39:14.249 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T11:39:14.250 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T11:39:14.251 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T11:39:14.252 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T11:39:14.253 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T11:39:14.254 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T11:39:14.255 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T11:39:14.256 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T11:39:14.257 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T11:39:14.258 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T11:39:14.259 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T11:39:14.260 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T11:39:14.260 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T11:39:14.261 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout:====================================================================================================================== 2023-10-06T11:39:14.262 INFO:teuthology.orchestra.run.smithi177.stdout:Install 118 Packages 2023-10-06T11:39:14.263 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 2 Packages 2023-10-06T11:39:14.263 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:39:14.263 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 201 M 2023-10-06T11:39:14.263 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-10-06T11:39:14.454 INFO:teuthology.orchestra.run.smithi177.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 71 kB/s | 6.4 kB 00:00 2023-10-06T11:39:14.616 INFO:teuthology.orchestra.run.smithi177.stdout:(2/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 5.3 MB/s | 877 kB 00:00 2023-10-06T11:39:14.658 INFO:teuthology.orchestra.run.smithi177.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 3.8 MB/s | 162 kB 00:00 2023-10-06T11:39:14.867 INFO:teuthology.orchestra.run.smithi177.stdout:(4/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 11 MB/s | 2.3 MB 00:00 2023-10-06T11:39:14.945 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T11:39:14.945 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T11:39:14.957 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T11:39:14.958 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T11:39:14.959 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T11:39:14.959 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T11:39:14.961 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T11:39:15.099 INFO:teuthology.orchestra.run.smithi177.stdout:(5/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 7.1 MB/s | 1.6 MB 00:00 2023-10-06T11:39:15.099 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T11:39:15.099 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T11:39:15.477 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-10-06T11:39:15.489 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================================================== 2023-10-06T11:39:15.489 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================================================== 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 6.4 k 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.2 M 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 877 k 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 162 k 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 1.6 M 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 121 k 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 1.7 M 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 7.4 M 2023-10-06T11:39:15.490 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 47 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 52 M 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 267 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 80 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 255 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 18 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 25 k 2023-10-06T11:39:15.491 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 797 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 130 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 190 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 371 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 358 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 107 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 91 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 173 k 2023-10-06T11:39:15.492 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.4 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.5 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 23 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 23 k 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 2.3 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 238 k 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 3.9 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 17 M 2023-10-06T11:39:15.493 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1274.g68e53a4a.el8 ceph-noarch 14 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 24 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 175 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T11:39:15.494 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T11:39:15.495 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T11:39:15.495 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 506 k 2023-10-06T11:39:15.495 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T11:39:15.495 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 5.6 M 2023-10-06T11:39:15.495 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T11:39:15.496 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T11:39:15.496 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T11:39:15.496 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T11:39:15.496 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T11:39:15.496 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 45 k 2023-10-06T11:39:15.497 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:17.2.6-1274.g68e53a4a.el8 ceph 115 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T11:39:15.498 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T11:39:15.499 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T11:39:15.500 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:39:15.501 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-10-06T11:39:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================================================== 2023-10-06T11:39:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:Install 118 Packages 2023-10-06T11:39:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2023-10-06T11:39:15.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:39:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 201 M 2023-10-06T11:39:15.503 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-10-06T11:39:15.609 INFO:teuthology.orchestra.run.smithi177.stdout:(6/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 7.5 MB/s | 3.9 MB 00:00 2023-10-06T11:39:15.799 INFO:teuthology.orchestra.run.smithi016.stdout:(1/120): ceph-17.2.6-1274.g68e53a4a.el8.x86_64. 28 kB/s | 6.4 kB 00:00 2023-10-06T11:39:15.801 INFO:teuthology.orchestra.run.smithi177.stdout:(7/120): ceph-common-17.2.6-1274.g68e53a4a.el8. 16 MB/s | 23 MB 00:01 2023-10-06T11:39:16.613 INFO:teuthology.orchestra.run.smithi016.stdout:(2/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x8 5.0 MB/s | 5.2 MB 00:01 2023-10-06T11:39:16.666 INFO:teuthology.orchestra.run.smithi016.stdout:(3/120): ceph-immutable-object-cache-17.2.6-127 2.9 MB/s | 162 kB 00:00 2023-10-06T11:39:16.870 INFO:teuthology.orchestra.run.smithi016.stdout:(4/120): ceph-fuse-17.2.6-1274.g68e53a4a.el8.x8 819 kB/s | 877 kB 00:01 2023-10-06T11:39:16.986 INFO:teuthology.orchestra.run.smithi016.stdout:(5/120): ceph-mds-17.2.6-1274.g68e53a4a.el8.x86 7.2 MB/s | 2.3 MB 00:00 2023-10-06T11:39:17.037 INFO:teuthology.orchestra.run.smithi177.stdout:(8/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el8 14 MB/s | 17 MB 00:01 2023-10-06T11:39:17.067 INFO:teuthology.orchestra.run.smithi177.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 817 kB/s | 24 kB 00:00 2023-10-06T11:39:17.485 INFO:teuthology.orchestra.run.smithi016.stdout:(6/120): ceph-mon-17.2.6-1274.g68e53a4a.el8.x86 7.9 MB/s | 3.9 MB 00:00 2023-10-06T11:39:17.817 INFO:teuthology.orchestra.run.smithi016.stdout:(7/120): ceph-mgr-17.2.6-1274.g68e53a4a.el8.x86 1.7 MB/s | 1.6 MB 00:00 2023-10-06T11:39:18.370 INFO:teuthology.orchestra.run.smithi177.stdout:(10/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x8 6.1 MB/s | 17 MB 00:02 2023-10-06T11:39:18.440 INFO:teuthology.orchestra.run.smithi177.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.6 MB/s | 255 kB 00:00 2023-10-06T11:39:18.507 INFO:teuthology.orchestra.run.smithi177.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 373 kB/s | 25 kB 00:00 2023-10-06T11:39:18.660 INFO:teuthology.orchestra.run.smithi177.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 5.1 MB/s | 797 kB 00:00 2023-10-06T11:39:18.730 INFO:teuthology.orchestra.run.smithi177.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 2.4 MB/s | 175 kB 00:00 2023-10-06T11:39:18.796 INFO:teuthology.orchestra.run.smithi177.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 1.9 MB/s | 130 kB 00:00 2023-10-06T11:39:18.940 INFO:teuthology.orchestra.run.smithi177.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 3.4 MB/s | 506 kB 00:00 2023-10-06T11:39:20.186 INFO:teuthology.orchestra.run.smithi177.stdout:(17/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 4.5 MB/s | 5.6 MB 00:01 2023-10-06T11:39:20.220 INFO:teuthology.orchestra.run.smithi016.stdout:(8/120): ceph-osd-17.2.6-1274.g68e53a4a.el8.x86 6.2 MB/s | 17 MB 00:02 2023-10-06T11:39:20.227 INFO:teuthology.orchestra.run.smithi177.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 1.1 MB/s | 45 kB 00:00 2023-10-06T11:39:20.236 INFO:teuthology.orchestra.run.smithi016.stdout:(9/120): ceph-selinux-17.2.6-1274.g68e53a4a.el8 597 kB/s | 24 kB 00:00 2023-10-06T11:39:20.280 INFO:teuthology.orchestra.run.smithi177.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 2.1 MB/s | 115 kB 00:00 2023-10-06T11:39:20.719 INFO:teuthology.orchestra.run.smithi177.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 2.6 MB/s | 190 kB 00:00 2023-10-06T11:39:20.721 INFO:teuthology.orchestra.run.smithi177.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 2.6 MB/s | 371 kB 00:00 2023-10-06T11:39:20.721 INFO:teuthology.orchestra.run.smithi177.stdout:(22/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 2.5 MB/s | 358 kB 00:00 2023-10-06T11:39:20.721 INFO:teuthology.orchestra.run.smithi177.stdout:(23/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 1.4 MB/s | 107 kB 00:00 2023-10-06T11:39:20.740 INFO:teuthology.orchestra.run.smithi177.stdout:(24/120): ceph-base-17.2.6-1274.g68e53a4a.el8.x 829 kB/s | 5.2 MB 00:06 2023-10-06T11:39:20.777 INFO:teuthology.orchestra.run.smithi177.stdout:(25/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 1.2 MB/s | 91 kB 00:00 2023-10-06T11:39:20.854 INFO:teuthology.orchestra.run.smithi177.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 2.2 MB/s | 173 kB 00:00 2023-10-06T11:39:20.926 INFO:teuthology.orchestra.run.smithi177.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 321 kB/s | 23 kB 00:00 2023-10-06T11:39:20.999 INFO:teuthology.orchestra.run.smithi177.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 1.6 MB/s | 121 kB 00:00 2023-10-06T11:39:21.565 INFO:teuthology.orchestra.run.smithi177.stdout:(29/120): ceph-test-17.2.6-1274.g68e53a4a.el8.x 11 MB/s | 52 MB 00:04 2023-10-06T11:39:21.593 INFO:teuthology.orchestra.run.smithi177.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 2.9 MB/s | 1.7 MB 00:00 2023-10-06T11:39:21.677 INFO:teuthology.orchestra.run.smithi177.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 2.3 MB/s | 238 kB 00:00 2023-10-06T11:39:21.732 INFO:teuthology.orchestra.run.smithi177.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 871 kB/s | 47 kB 00:00 2023-10-06T11:39:21.787 INFO:teuthology.orchestra.run.smithi177.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 263 kB/s | 14 kB 00:00 2023-10-06T11:39:21.904 INFO:teuthology.orchestra.run.smithi177.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 2.2 MB/s | 267 kB 00:00 2023-10-06T11:39:21.960 INFO:teuthology.orchestra.run.smithi177.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.4 MB/s | 80 kB 00:00 2023-10-06T11:39:22.015 INFO:teuthology.orchestra.run.smithi177.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 324 kB/s | 18 kB 00:00 2023-10-06T11:39:22.340 INFO:teuthology.orchestra.run.smithi177.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 9.5 MB/s | 7.4 MB 00:00 2023-10-06T11:39:22.444 INFO:teuthology.orchestra.run.smithi177.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 714 kB/s | 306 kB 00:00 2023-10-06T11:39:22.509 INFO:teuthology.orchestra.run.smithi177.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-06T11:39:22.622 INFO:teuthology.orchestra.run.smithi177.stdout:(40/120): liboath-2.6.2-3.el8.x86_64.rpm 208 kB/s | 59 kB 00:00 2023-10-06T11:39:22.630 INFO:teuthology.orchestra.run.smithi177.stdout:(41/120): python3-asyncssh-2.7.0-2.el8.noarch.r 3.9 MB/s | 476 kB 00:00 2023-10-06T11:39:22.661 INFO:teuthology.orchestra.run.smithi016.stdout:(10/120): ceph-common-17.2.6-1274.g68e53a4a.el8 3.3 MB/s | 23 MB 00:07 2023-10-06T11:39:22.697 INFO:teuthology.orchestra.run.smithi177.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-10-06T11:39:22.702 INFO:teuthology.orchestra.run.smithi177.stdout:(43/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 547 kB/s | 44 kB 00:00 2023-10-06T11:39:22.730 INFO:teuthology.orchestra.run.smithi016.stdout:(11/120): cephfs-mirror-17.2.6-1274.g68e53a4a.e 3.7 MB/s | 255 kB 00:00 2023-10-06T11:39:22.758 INFO:teuthology.orchestra.run.smithi177.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 549 kB/s | 33 kB 00:00 2023-10-06T11:39:22.765 INFO:teuthology.orchestra.run.smithi177.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 261 kB/s | 16 kB 00:00 2023-10-06T11:39:22.770 INFO:teuthology.orchestra.run.smithi016.stdout:(12/120): libcephfs-devel-17.2.6-1274.g68e53a4a 678 kB/s | 25 kB 00:00 2023-10-06T11:39:22.823 INFO:teuthology.orchestra.run.smithi177.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 2.6 MB/s | 173 kB 00:00 2023-10-06T11:39:22.861 INFO:teuthology.orchestra.run.smithi016.stdout:(13/120): libcephfs2-17.2.6-1274.g68e53a4a.el8. 8.3 MB/s | 797 kB 00:00 2023-10-06T11:39:22.882 INFO:teuthology.orchestra.run.smithi177.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 1.3 MB/s | 81 kB 00:00 2023-10-06T11:39:22.903 INFO:teuthology.orchestra.run.smithi016.stdout:(14/120): libcephsqlite-17.2.6-1274.g68e53a4a.e 4.2 MB/s | 175 kB 00:00 2023-10-06T11:39:22.938 INFO:teuthology.orchestra.run.smithi177.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 200 kB/s | 11 kB 00:00 2023-10-06T11:39:22.942 INFO:teuthology.orchestra.run.smithi016.stdout:(15/120): librados-devel-17.2.6-1274.g68e53a4a. 3.2 MB/s | 130 kB 00:00 2023-10-06T11:39:22.958 INFO:teuthology.orchestra.run.smithi177.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2023-10-06T11:39:22.994 INFO:teuthology.orchestra.run.smithi177.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 315 kB/s | 18 kB 00:00 2023-10-06T11:39:22.997 INFO:teuthology.orchestra.run.smithi016.stdout:(16/120): libradosstriper1-17.2.6-1274.g68e53a4 9.0 MB/s | 506 kB 00:00 2023-10-06T11:39:23.051 INFO:teuthology.orchestra.run.smithi177.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 856 kB/s | 49 kB 00:00 2023-10-06T11:39:23.109 INFO:teuthology.orchestra.run.smithi177.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-06T11:39:23.129 INFO:teuthology.orchestra.run.smithi177.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 6.8 MB/s | 1.2 MB 00:00 2023-10-06T11:39:23.166 INFO:teuthology.orchestra.run.smithi177.stdout:(54/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 1.4 MB/s | 3.5 MB 00:02 2023-10-06T11:39:23.167 INFO:teuthology.orchestra.run.smithi177.stdout:(55/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-10-06T11:39:23.192 INFO:teuthology.orchestra.run.smithi177.stdout:(56/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.4 MB/s | 283 kB 00:00 2023-10-06T11:39:23.227 INFO:teuthology.orchestra.run.smithi177.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 567 kB/s | 34 kB 00:00 2023-10-06T11:39:23.245 INFO:teuthology.orchestra.run.smithi016.stdout:(17/120): ceph-radosgw-17.2.6-1274.g68e53a4a.el 3.1 MB/s | 17 MB 00:05 2023-10-06T11:39:23.252 INFO:teuthology.orchestra.run.smithi177.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 3.2 MB/s | 196 kB 00:00 2023-10-06T11:39:23.285 INFO:teuthology.orchestra.run.smithi177.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 1.1 MB/s | 64 kB 00:00 2023-10-06T11:39:23.303 INFO:teuthology.orchestra.run.smithi016.stdout:(18/120): python3-ceph-argparse-17.2.6-1274.g68 779 kB/s | 45 kB 00:00 2023-10-06T11:39:23.307 INFO:teuthology.orchestra.run.smithi177.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 340 kB/s | 19 kB 00:00 2023-10-06T11:39:23.342 INFO:teuthology.orchestra.run.smithi177.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 424 kB/s | 24 kB 00:00 2023-10-06T11:39:23.353 INFO:teuthology.orchestra.run.smithi177.stdout:(62/120): python3-portend-2.6-1.el8.noarch.rpm 83 kB/s | 16 kB 00:00 2023-10-06T11:39:23.354 INFO:teuthology.orchestra.run.smithi016.stdout:(19/120): python3-ceph-common-17.2.6-1274.g68e5 2.3 MB/s | 115 kB 00:00 2023-10-06T11:39:23.363 INFO:teuthology.orchestra.run.smithi177.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 528 kB/s | 29 kB 00:00 2023-10-06T11:39:23.398 INFO:teuthology.orchestra.run.smithi177.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 494 kB/s | 27 kB 00:00 2023-10-06T11:39:23.402 INFO:teuthology.orchestra.run.smithi016.stdout:(20/120): python3-cephfs-17.2.6-1274.g68e53a4a. 3.8 MB/s | 190 kB 00:00 2023-10-06T11:39:23.425 INFO:teuthology.orchestra.run.smithi177.stdout:(65/120): python3-webob-1.8.5-1.el8.1.noarch.rp 4.0 MB/s | 251 kB 00:00 2023-10-06T11:39:23.456 INFO:teuthology.orchestra.run.smithi177.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-06T11:39:23.486 INFO:teuthology.orchestra.run.smithi177.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-10-06T11:39:23.496 INFO:teuthology.orchestra.run.smithi016.stdout:(21/120): python3-rados-17.2.6-1274.g68e53a4a.e 3.9 MB/s | 371 kB 00:00 2023-10-06T11:39:23.512 INFO:teuthology.orchestra.run.smithi177.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 412 kB/s | 23 kB 00:00 2023-10-06T11:39:23.516 INFO:teuthology.orchestra.run.smithi177.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-06T11:39:23.533 INFO:teuthology.orchestra.run.smithi016.stdout:(22/120): librgw2-17.2.6-1274.g68e53a4a.el8.x86 10 MB/s | 5.6 MB 00:00 2023-10-06T11:39:23.579 INFO:teuthology.orchestra.run.smithi016.stdout:(23/120): python3-rbd-17.2.6-1274.g68e53a4a.el8 4.1 MB/s | 358 kB 00:00 2023-10-06T11:39:23.582 INFO:teuthology.orchestra.run.smithi016.stdout:(24/120): python3-rgw-17.2.6-1274.g68e53a4a.el8 2.1 MB/s | 107 kB 00:00 2023-10-06T11:39:23.593 INFO:teuthology.orchestra.run.smithi177.stdout:(70/120): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2023-10-06T11:39:23.598 INFO:teuthology.orchestra.run.smithi177.stdout:(71/120): python3-waitress-1.4.3-1.el8.noarch.r 1.0 MB/s | 255 kB 00:00 2023-10-06T11:39:23.617 INFO:teuthology.orchestra.run.smithi016.stdout:(25/120): rbd-fuse-17.2.6-1274.g68e53a4a.el8.x8 2.4 MB/s | 91 kB 00:00 2023-10-06T11:39:23.628 INFO:teuthology.orchestra.run.smithi177.stdout:(72/120): python3-prettytable-0.7.2-14.el8.noar 1.2 MB/s | 44 kB 00:00 2023-10-06T11:39:23.632 INFO:teuthology.orchestra.run.smithi177.stdout:(73/120): python3-jsonpatch-1.21-2.el8.noarch.r 719 kB/s | 27 kB 00:00 2023-10-06T11:39:23.647 INFO:teuthology.orchestra.run.smithi177.stdout:(74/120): jq-1.5-12.el8.x86_64.rpm 8.3 MB/s | 161 kB 00:00 2023-10-06T11:39:23.653 INFO:teuthology.orchestra.run.smithi177.stdout:(75/120): flex-2.6.1-9.el8.x86_64.rpm 15 MB/s | 320 kB 00:00 2023-10-06T11:39:23.655 INFO:teuthology.orchestra.run.smithi016.stdout:(26/120): rbd-nbd-17.2.6-1274.g68e53a4a.el8.x86 4.4 MB/s | 173 kB 00:00 2023-10-06T11:39:23.670 INFO:teuthology.orchestra.run.smithi177.stdout:(76/120): python3-werkzeug-0.12.2-4.el8.noarch. 19 MB/s | 457 kB 00:00 2023-10-06T11:39:23.686 INFO:teuthology.orchestra.run.smithi177.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.5 MB/s | 39 kB 00:00 2023-10-06T11:39:23.697 INFO:teuthology.orchestra.run.smithi016.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1274.g 548 kB/s | 23 kB 00:00 2023-10-06T11:39:23.711 INFO:teuthology.orchestra.run.smithi177.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 27 MB/s | 688 kB 00:00 2023-10-06T11:39:23.730 INFO:teuthology.orchestra.run.smithi177.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 14 MB/s | 259 kB 00:00 2023-10-06T11:39:23.742 INFO:teuthology.orchestra.run.smithi016.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1274.g68e53a4 2.6 MB/s | 121 kB 00:00 2023-10-06T11:39:23.747 INFO:teuthology.orchestra.run.smithi177.stdout:(80/120): python3-babel-2.5.1-5.el8.noarch.rpm 51 MB/s | 4.8 MB 00:00 2023-10-06T11:39:23.751 INFO:teuthology.orchestra.run.smithi177.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 11 MB/s | 225 kB 00:00 2023-10-06T11:39:23.860 INFO:teuthology.orchestra.run.smithi177.stdout:(82/120): python3-scipy-1.0.0-20.module+el8.1.0 42 MB/s | 15 MB 00:00 2023-10-06T11:39:23.863 INFO:teuthology.orchestra.run.smithi177.stdout:(83/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-10-06T11:39:23.876 INFO:teuthology.orchestra.run.smithi177.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-06T11:39:23.884 INFO:teuthology.orchestra.run.smithi177.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 26 MB/s | 538 kB 00:00 2023-10-06T11:39:23.892 INFO:teuthology.orchestra.run.smithi177.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-10-06T11:39:23.899 INFO:teuthology.orchestra.run.smithi177.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 10 MB/s | 157 kB 00:00 2023-10-06T11:39:23.906 INFO:teuthology.orchestra.run.smithi016.stdout:(29/120): rbd-mirror-17.2.6-1274.g68e53a4a.el8. 11 MB/s | 3.5 MB 00:00 2023-10-06T11:39:23.911 INFO:teuthology.orchestra.run.smithi177.stdout:(88/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 23 MB/s | 3.7 MB 00:00 2023-10-06T11:39:23.913 INFO:teuthology.orchestra.run.smithi177.stdout:(89/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.1 MB/s | 110 kB 00:00 2023-10-06T11:39:23.916 INFO:teuthology.orchestra.run.smithi177.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T11:39:23.925 INFO:teuthology.orchestra.run.smithi177.stdout:(91/120): python3-jsonpointer-1.10-11.el8.noarc 1.5 MB/s | 20 kB 00:00 2023-10-06T11:39:23.931 INFO:teuthology.orchestra.run.smithi177.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.6 MB/s | 54 kB 00:00 2023-10-06T11:39:23.935 INFO:teuthology.orchestra.run.smithi177.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 32 MB/s | 709 kB 00:00 2023-10-06T11:39:23.961 INFO:teuthology.orchestra.run.smithi177.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 28 MB/s | 713 kB 00:00 2023-10-06T11:39:23.974 INFO:teuthology.orchestra.run.smithi177.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.4 MB/s | 19 kB 00:00 2023-10-06T11:39:23.986 INFO:teuthology.orchestra.run.smithi177.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.5 MB/s | 43 kB 00:00 2023-10-06T11:39:24.010 INFO:teuthology.orchestra.run.smithi177.stdout:(97/120): openblas-threads-0.3.12-1.el8.x86_64. 58 MB/s | 4.7 MB 00:00 2023-10-06T11:39:24.010 INFO:teuthology.orchestra.run.smithi177.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 6.5 MB/s | 155 kB 00:00 2023-10-06T11:39:24.025 INFO:teuthology.orchestra.run.smithi177.stdout:(99/120): openblas-0.3.12-1.el8.x86_64.rpm 50 MB/s | 4.6 MB 00:00 2023-10-06T11:39:24.026 INFO:teuthology.orchestra.run.smithi177.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 3.0 MB/s | 48 kB 00:00 2023-10-06T11:39:24.027 INFO:teuthology.orchestra.run.smithi177.stdout:(101/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 11 MB/s | 220 kB 00:00 2023-10-06T11:39:24.039 INFO:teuthology.orchestra.run.smithi177.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 3.2 MB/s | 43 kB 00:00 2023-10-06T11:39:24.043 INFO:teuthology.orchestra.run.smithi177.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-10-06T11:39:24.052 INFO:teuthology.orchestra.run.smithi177.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.4 MB/s | 18 kB 00:00 2023-10-06T11:39:24.070 INFO:teuthology.orchestra.run.smithi177.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 22 MB/s | 605 kB 00:00 2023-10-06T11:39:24.071 INFO:teuthology.orchestra.run.smithi177.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 4.1 MB/s | 83 kB 00:00 2023-10-06T11:39:24.080 INFO:teuthology.orchestra.run.smithi177.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 44 MB/s | 2.3 MB 00:00 2023-10-06T11:39:24.085 INFO:teuthology.orchestra.run.smithi177.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 4.0 MB/s | 59 kB 00:00 2023-10-06T11:39:24.088 INFO:teuthology.orchestra.run.smithi177.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-06T11:39:24.096 INFO:teuthology.orchestra.run.smithi177.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.5 MB/s | 38 kB 00:00 2023-10-06T11:39:24.099 INFO:teuthology.orchestra.run.smithi177.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-10-06T11:39:24.101 INFO:teuthology.orchestra.run.smithi177.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.3 MB/s | 58 kB 00:00 2023-10-06T11:39:24.110 INFO:teuthology.orchestra.run.smithi177.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.4 MB/s | 47 kB 00:00 2023-10-06T11:39:24.117 INFO:teuthology.orchestra.run.smithi016.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1274.g68e53 4.5 MB/s | 1.7 MB 00:00 2023-10-06T11:39:24.120 INFO:teuthology.orchestra.run.smithi177.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 27 MB/s | 559 kB 00:00 2023-10-06T11:39:24.123 INFO:teuthology.orchestra.run.smithi177.stdout:(115/120): libgfortran-8.4.1-1.el8.x86_64.rpm 29 MB/s | 639 kB 00:00 2023-10-06T11:39:24.124 INFO:teuthology.orchestra.run.smithi177.stdout:(116/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.8 MB/s | 70 kB 00:00 2023-10-06T11:39:24.134 INFO:teuthology.orchestra.run.smithi177.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-10-06T11:39:24.136 INFO:teuthology.orchestra.run.smithi177.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.3 MB/s | 97 kB 00:00 2023-10-06T11:39:24.204 INFO:teuthology.orchestra.run.smithi016.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1274.g68 2.7 MB/s | 238 kB 00:00 2023-10-06T11:39:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:(32/120): ceph-mgr-rook-17.2.6-1274.g68e53a4a.e 990 kB/s | 47 kB 00:00 2023-10-06T11:39:24.305 INFO:teuthology.orchestra.run.smithi016.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1274.g6 272 kB/s | 14 kB 00:00 2023-10-06T11:39:24.418 INFO:teuthology.orchestra.run.smithi016.stdout:(34/120): ceph-volume-17.2.6-1274.g68e53a4a.el8 2.3 MB/s | 267 kB 00:00 2023-10-06T11:39:24.476 INFO:teuthology.orchestra.run.smithi016.stdout:(35/120): cephadm-17.2.6-1274.g68e53a4a.el8.noa 1.4 MB/s | 80 kB 00:00 2023-10-06T11:39:24.531 INFO:teuthology.orchestra.run.smithi016.stdout:(36/120): cephfs-top-17.2.6-1274.g68e53a4a.el8. 322 kB/s | 18 kB 00:00 2023-10-06T11:39:24.614 INFO:teuthology.orchestra.run.smithi016.stdout:(37/120): ceph-mgr-diskprediction-local-17.2.6- 10 MB/s | 7.4 MB 00:00 2023-10-06T11:39:24.734 INFO:teuthology.orchestra.run.smithi177.stdout:(119/120): librados2-17.2.6-1274.g68e53a4a.el8. 5.6 MB/s | 3.4 MB 00:00 2023-10-06T11:39:25.280 INFO:teuthology.orchestra.run.smithi016.stdout:(38/120): ceph-test-17.2.6-1274.g68e53a4a.el8.x 10 MB/s | 52 MB 00:05 2023-10-06T11:39:25.280 INFO:teuthology.orchestra.run.smithi016.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 88 kB/s | 59 kB 00:00 2023-10-06T11:39:25.283 INFO:teuthology.orchestra.run.smithi016.stdout:(40/120): gperftools-libs-2.7-9.el8.x86_64.rpm 407 kB/s | 306 kB 00:00 2023-10-06T11:39:25.365 INFO:teuthology.orchestra.run.smithi016.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 533 kB/s | 44 kB 00:00 2023-10-06T11:39:25.449 INFO:teuthology.orchestra.run.smithi016.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-10-06T11:39:25.530 INFO:teuthology.orchestra.run.smithi016.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 406 kB/s | 33 kB 00:00 2023-10-06T11:39:25.582 INFO:teuthology.orchestra.run.smithi177.stdout:(120/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 2.5 MB/s | 3.5 MB 00:01 2023-10-06T11:39:25.587 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-10-06T11:39:25.587 INFO:teuthology.orchestra.run.smithi177.stdout:Total 18 MB/s | 201 MB 00:11 2023-10-06T11:39:25.602 INFO:teuthology.orchestra.run.smithi016.stdout:(44/120): libunwind-1.3.1-3.el8.x86_64.rpm 233 kB/s | 75 kB 00:00 2023-10-06T11:39:25.612 INFO:teuthology.orchestra.run.smithi016.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 201 kB/s | 16 kB 00:00 2023-10-06T11:39:25.636 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-10-06T11:39:25.686 INFO:teuthology.orchestra.run.smithi016.stdout:(46/120): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-10-06T11:39:25.748 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-10-06T11:39:25.748 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-10-06T11:39:25.766 INFO:teuthology.orchestra.run.smithi016.stdout:(47/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-10-06T11:39:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 996 kB/s | 81 kB 00:00 2023-10-06T11:39:25.775 INFO:teuthology.orchestra.run.smithi016.stdout:(49/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-10-06T11:39:25.847 INFO:teuthology.orchestra.run.smithi016.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 137 kB/s | 11 kB 00:00 2023-10-06T11:39:25.849 INFO:teuthology.orchestra.run.smithi016.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 216 kB/s | 18 kB 00:00 2023-10-06T11:39:25.929 INFO:teuthology.orchestra.run.smithi016.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 598 kB/s | 49 kB 00:00 2023-10-06T11:39:25.931 INFO:teuthology.orchestra.run.smithi016.stdout:(53/120): python3-more-itertools-7.2.0-3.el8.no 720 kB/s | 59 kB 00:00 2023-10-06T11:39:26.010 INFO:teuthology.orchestra.run.smithi016.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 740 kB/s | 60 kB 00:00 2023-10-06T11:39:26.014 INFO:teuthology.orchestra.run.smithi016.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-06T11:39:26.021 INFO:teuthology.orchestra.run.smithi016.stdout:(56/120): python3-kubernetes-11.0.0-6.el8.noarc 4.7 MB/s | 1.2 MB 00:00 2023-10-06T11:39:26.091 INFO:teuthology.orchestra.run.smithi016.stdout:(57/120): python3-portend-2.6-1.el8.noarch.rpm 193 kB/s | 16 kB 00:00 2023-10-06T11:39:26.096 INFO:teuthology.orchestra.run.smithi016.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 414 kB/s | 34 kB 00:00 2023-10-06T11:39:26.104 INFO:teuthology.orchestra.run.smithi016.stdout:(59/120): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-10-06T11:39:26.173 INFO:teuthology.orchestra.run.smithi016.stdout:(60/120): python3-rsa-4.9-2.el8.noarch.rpm 788 kB/s | 64 kB 00:00 2023-10-06T11:39:26.177 INFO:teuthology.orchestra.run.smithi016.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 231 kB/s | 19 kB 00:00 2023-10-06T11:39:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 294 kB/s | 24 kB 00:00 2023-10-06T11:39:26.254 INFO:teuthology.orchestra.run.smithi016.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 362 kB/s | 29 kB 00:00 2023-10-06T11:39:26.258 INFO:teuthology.orchestra.run.smithi016.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 340 kB/s | 27 kB 00:00 2023-10-06T11:39:26.270 INFO:teuthology.orchestra.run.smithi016.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-10-06T11:39:26.340 INFO:teuthology.orchestra.run.smithi016.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 752 kB/s | 61 kB 00:00 2023-10-06T11:39:26.352 INFO:teuthology.orchestra.run.smithi016.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-06T11:39:26.438 INFO:teuthology.orchestra.run.smithi016.stdout:(68/120): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-10-06T11:39:26.505 INFO:teuthology.orchestra.run.smithi016.stdout:(69/120): python3-zc-lockfile-2.0-2.el8.noarch. 139 kB/s | 23 kB 00:00 2023-10-06T11:39:26.509 INFO:teuthology.orchestra.run.smithi016.stdout:(70/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 983 kB/s | 70 kB 00:00 2023-10-06T11:39:26.549 INFO:teuthology.orchestra.run.smithi016.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.1 MB/s | 44 kB 00:00 2023-10-06T11:39:26.563 INFO:teuthology.orchestra.run.smithi016.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-06T11:39:26.580 INFO:teuthology.orchestra.run.smithi016.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 9.5 MB/s | 161 kB 00:00 2023-10-06T11:39:26.601 INFO:teuthology.orchestra.run.smithi016.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 15 MB/s | 320 kB 00:00 2023-10-06T11:39:26.624 INFO:teuthology.orchestra.run.smithi016.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 20 MB/s | 457 kB 00:00 2023-10-06T11:39:26.685 INFO:teuthology.orchestra.run.smithi016.stdout:(76/120): thrift-0.13.0-2.el8.x86_64.rpm 5.2 MB/s | 1.7 MB 00:00 2023-10-06T11:39:26.716 INFO:teuthology.orchestra.run.smithi016.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 1.2 MB/s | 39 kB 00:00 2023-10-06T11:39:26.738 INFO:teuthology.orchestra.run.smithi016.stdout:(78/120): python3-babel-2.5.1-5.el8.noarch.rpm 42 MB/s | 4.8 MB 00:00 2023-10-06T11:39:26.750 INFO:teuthology.orchestra.run.smithi016.stdout:(79/120): bison-3.0.4-10.el8.x86_64.rpm 20 MB/s | 688 kB 00:00 2023-10-06T11:39:26.755 INFO:teuthology.orchestra.run.smithi016.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-06T11:39:26.768 INFO:teuthology.orchestra.run.smithi016.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 13 MB/s | 225 kB 00:00 2023-10-06T11:39:26.771 INFO:teuthology.orchestra.run.smithi016.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 8.1 MB/s | 126 kB 00:00 2023-10-06T11:39:26.900 INFO:teuthology.orchestra.run.smithi016.stdout:(83/120): python3-scipy-1.0.0-20.module+el8.1.0 37 MB/s | 15 MB 00:00 2023-10-06T11:39:26.904 INFO:teuthology.orchestra.run.smithi016.stdout:(84/120): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-10-06T11:39:26.926 INFO:teuthology.orchestra.run.smithi016.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 21 MB/s | 538 kB 00:00 2023-10-06T11:39:26.930 INFO:teuthology.orchestra.run.smithi016.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 11 MB/s | 302 kB 00:00 2023-10-06T11:39:26.940 INFO:teuthology.orchestra.run.smithi016.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-10-06T11:39:26.944 INFO:teuthology.orchestra.run.smithi016.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 7.6 MB/s | 110 kB 00:00 2023-10-06T11:39:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:(89/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 19 MB/s | 3.7 MB 00:00 2023-10-06T11:39:26.963 INFO:teuthology.orchestra.run.smithi016.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.0 MB/s | 20 kB 00:00 2023-10-06T11:39:26.968 INFO:teuthology.orchestra.run.smithi016.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 13 MB/s | 353 kB 00:00 2023-10-06T11:39:26.979 INFO:teuthology.orchestra.run.smithi016.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.5 MB/s | 54 kB 00:00 2023-10-06T11:39:26.988 INFO:teuthology.orchestra.run.smithi016.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 28 MB/s | 709 kB 00:00 2023-10-06T11:39:27.015 INFO:teuthology.orchestra.run.smithi016.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 26 MB/s | 713 kB 00:00 2023-10-06T11:39:27.028 INFO:teuthology.orchestra.run.smithi016.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.5 MB/s | 19 kB 00:00 2023-10-06T11:39:27.040 INFO:teuthology.orchestra.run.smithi016.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.5 MB/s | 43 kB 00:00 2023-10-06T11:39:27.054 INFO:teuthology.orchestra.run.smithi016.stdout:(97/120): python3-oauthlib-2.1.0-1.el8.noarch.r 11 MB/s | 155 kB 00:00 2023-10-06T11:39:27.069 INFO:teuthology.orchestra.run.smithi016.stdout:(98/120): openblas-threads-0.3.12-1.el8.x86_64. 47 MB/s | 4.7 MB 00:00 2023-10-06T11:39:27.072 INFO:teuthology.orchestra.run.smithi016.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 12 MB/s | 220 kB 00:00 2023-10-06T11:39:27.089 INFO:teuthology.orchestra.run.smithi016.stdout:(100/120): openblas-0.3.12-1.el8.x86_64.rpm 42 MB/s | 4.6 MB 00:00 2023-10-06T11:39:27.090 INFO:teuthology.orchestra.run.smithi016.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.2 MB/s | 48 kB 00:00 2023-10-06T11:39:27.093 INFO:teuthology.orchestra.run.smithi016.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 11 MB/s | 229 kB 00:00 2023-10-06T11:39:27.102 INFO:teuthology.orchestra.run.smithi016.stdout:(103/120): python3-requests-oauthlib-1.0.0-1.el 3.3 MB/s | 43 kB 00:00 2023-10-06T11:39:27.106 INFO:teuthology.orchestra.run.smithi016.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.4 MB/s | 18 kB 00:00 2023-10-06T11:39:27.120 INFO:teuthology.orchestra.run.smithi016.stdout:(105/120): fuse-2.9.7-12.el8.x86_64.rpm 5.8 MB/s | 83 kB 00:00 2023-10-06T11:39:27.125 INFO:teuthology.orchestra.run.smithi016.stdout:(106/120): pcre2-devel-10.32-2.el8.x86_64.rpm 25 MB/s | 605 kB 00:00 2023-10-06T11:39:27.132 INFO:teuthology.orchestra.run.smithi016.stdout:(107/120): elfutils-libelf-devel-0.182-3.el8.x8 4.7 MB/s | 59 kB 00:00 2023-10-06T11:39:27.139 INFO:teuthology.orchestra.run.smithi016.stdout:(108/120): openssl-devel-1.1.1g-15.el8_3.x86_64 47 MB/s | 2.3 MB 00:00 2023-10-06T11:39:27.142 INFO:teuthology.orchestra.run.smithi016.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-06T11:39:27.145 INFO:teuthology.orchestra.run.smithi016.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.9 MB/s | 38 kB 00:00 2023-10-06T11:39:27.154 INFO:teuthology.orchestra.run.smithi016.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.8 MB/s | 86 kB 00:00 2023-10-06T11:39:27.157 INFO:teuthology.orchestra.run.smithi016.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.7 MB/s | 58 kB 00:00 2023-10-06T11:39:27.160 INFO:teuthology.orchestra.run.smithi016.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.1 MB/s | 47 kB 00:00 2023-10-06T11:39:27.175 INFO:teuthology.orchestra.run.smithi016.stdout:(114/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.7 MB/s | 70 kB 00:00 2023-10-06T11:39:27.177 INFO:teuthology.orchestra.run.smithi016.stdout:(115/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 23 MB/s | 559 kB 00:00 2023-10-06T11:39:27.183 INFO:teuthology.orchestra.run.smithi016.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 24 MB/s | 639 kB 00:00 2023-10-06T11:39:27.190 INFO:teuthology.orchestra.run.smithi016.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-10-06T11:39:27.191 INFO:teuthology.orchestra.run.smithi016.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.0 MB/s | 97 kB 00:00 2023-10-06T11:39:27.462 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-10-06T11:39:27.467 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-10-06T11:39:27.470 INFO:teuthology.orchestra.run.smithi016.stdout:(119/120): librbd1-17.2.6-1274.g68e53a4a.el8.x8 13 MB/s | 3.5 MB 00:00 2023-10-06T11:39:27.842 INFO:teuthology.orchestra.run.smithi016.stdout:(120/120): librados2-17.2.6-1274.g68e53a4a.el8. 5.2 MB/s | 3.4 MB 00:00 2023-10-06T11:39:27.845 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-10-06T11:39:27.845 INFO:teuthology.orchestra.run.smithi016.stdout:Total 16 MB/s | 201 MB 00:12 2023-10-06T11:39:27.897 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-10-06T11:39:28.021 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-10-06T11:39:28.021 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-10-06T11:39:29.745 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-10-06T11:39:29.749 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-10-06T11:39:31.933 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-10-06T11:39:32.123 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T11:39:32.310 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T11:39:32.510 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T11:39:32.636 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T11:39:32.852 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T11:39:33.221 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T11:39:33.460 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T11:39:33.862 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T11:39:34.043 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:39:34.192 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:39:34.376 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:39:34.477 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:39:34.815 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T11:39:35.135 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-10-06T11:39:35.320 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T11:39:35.495 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T11:39:35.557 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T11:39:35.668 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:39:35.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T11:39:35.769 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:39:35.879 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T11:39:35.997 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T11:39:36.071 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T11:39:36.158 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T11:39:36.343 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:39:36.461 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:39:36.477 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T11:39:36.630 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T11:39:36.708 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T11:39:36.729 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T11:39:36.854 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T11:39:37.010 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T11:39:37.139 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 5/122 2023-10-06T11:39:37.146 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T11:39:37.317 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T11:39:37.874 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T11:39:38.012 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:39:38.067 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T11:39:38.146 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:39:38.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:39:38.469 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:39:38.605 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T11:39:38.784 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T11:39:38.828 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T11:39:39.511 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T11:39:39.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T11:39:39.737 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:39:39.762 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T11:39:39.855 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:39:39.966 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T11:39:40.129 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T11:39:40.217 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T11:39:40.370 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T11:39:40.373 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T11:39:40.595 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:39:40.660 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T11:39:40.696 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:39:40.853 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T11:39:40.912 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T11:39:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 14/122 2023-10-06T11:39:41.090 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T11:39:41.102 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T11:39:41.219 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T11:39:41.270 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T11:39:41.417 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T11:39:41.464 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T11:39:41.603 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T11:39:42.187 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T11:39:42.345 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T11:39:42.513 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T11:39:42.873 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T11:39:42.893 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T11:39:43.086 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T11:39:43.312 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T11:39:43.548 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T11:39:43.808 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T11:39:43.859 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T11:39:44.025 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T11:39:44.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T11:39:44.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T11:39:44.476 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T11:39:44.725 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T11:39:45.001 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T11:39:45.252 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T11:39:45.452 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 26/122 2023-10-06T11:39:45.561 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 27/122 2023-10-06T11:39:45.806 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 28/122 2023-10-06T11:39:46.088 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T11:39:46.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T11:39:46.655 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T11:39:46.886 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T11:39:46.902 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T11:39:47.092 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T11:39:47.193 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T11:39:47.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T11:39:47.443 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T11:39:47.501 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T11:39:47.616 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T11:39:47.769 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T11:39:47.813 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T11:39:47.976 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T11:39:48.041 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T11:39:48.135 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T11:39:48.294 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T11:39:48.638 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T11:39:48.817 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T11:39:48.985 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T11:39:49.281 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T11:39:49.579 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T11:39:49.823 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T11:39:50.162 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T11:39:50.397 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T11:39:50.499 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T11:39:50.597 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T11:39:50.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T11:39:50.850 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T11:39:50.976 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T11:39:51.106 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T11:39:51.165 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T11:39:51.256 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T11:39:51.281 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T11:39:51.423 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T11:39:51.506 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 38/122 2023-10-06T11:39:51.592 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T11:39:51.689 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 39/122 2023-10-06T11:39:51.757 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T11:39:51.897 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 40/122 2023-10-06T11:39:51.953 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T11:39:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T11:39:52.209 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T11:39:52.230 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T11:39:52.413 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T11:39:52.448 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T11:39:52.580 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T11:39:52.688 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T11:39:52.861 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T11:39:52.876 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T11:39:53.009 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T11:39:53.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T11:39:53.190 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T11:39:53.266 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T11:39:53.385 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T11:39:53.601 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T11:39:53.643 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T11:39:53.818 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T11:39:53.885 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T11:39:53.995 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T11:39:54.172 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T11:39:54.239 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T11:39:54.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T11:39:54.517 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T11:39:54.670 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T11:39:54.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T11:39:54.903 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T11:39:54.954 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T11:39:55.077 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T11:39:55.205 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T11:39:55.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T11:39:55.413 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T11:39:55.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T11:39:55.536 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T11:39:55.663 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T11:39:55.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T11:39:55.891 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T11:39:56.053 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T11:39:56.261 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T11:39:56.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T11:39:56.806 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T11:39:56.846 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T11:39:57.036 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T11:39:57.120 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T11:39:57.215 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T11:39:57.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T11:39:57.357 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T11:39:57.561 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T11:39:57.690 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T11:39:57.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T11:39:57.969 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T11:39:57.973 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T11:39:58.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T11:39:58.320 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T11:39:58.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T11:39:58.515 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T11:39:58.570 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T11:39:58.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T11:39:58.800 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T11:39:58.989 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T11:39:58.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T11:39:59.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T11:39:59.568 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T11:39:59.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T11:39:59.979 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T11:40:01.242 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T11:40:01.551 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T11:40:01.788 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T11:40:02.120 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T11:40:02.388 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T11:40:02.750 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T11:40:03.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T11:40:03.212 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T11:40:03.434 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T11:40:03.934 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:04.505 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:05.089 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:05.374 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T11:40:05.475 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T11:40:05.629 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T11:40:08.874 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:09.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:09.793 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 87/122 2023-10-06T11:40:10.602 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T11:40:10.728 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 88/122 2023-10-06T11:40:10.858 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T11:40:23.562 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T11:40:23.562 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2023-10-06T11:40:23.562 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2023-10-06T11:40:23.562 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-10-06T11:40:23.562 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /run 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-10-06T11:40:23.563 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:40:23.934 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T11:40:25.430 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T11:40:25.620 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T11:40:25.842 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T11:40:25.976 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T11:40:26.377 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T11:40:26.559 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T11:40:26.576 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T11:40:29.449 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 89/122 2023-10-06T11:40:29.449 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-10-06T11:40:29.449 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-10-06T11:40:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:40:29.460 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T11:40:29.639 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T11:40:29.798 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T11:40:29.806 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T11:40:30.004 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T11:40:30.166 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T11:40:30.383 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T11:40:30.594 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T11:40:31.382 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 90/122 2023-10-06T11:40:31.549 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T11:40:31.564 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T11:40:31.749 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T11:40:31.783 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 91/122 2023-10-06T11:40:31.849 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T11:40:31.939 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T11:40:32.192 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T11:40:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 92/122 2023-10-06T11:40:32.377 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T11:40:32.416 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T11:40:32.530 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T11:40:32.548 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 93/122 2023-10-06T11:40:32.591 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T11:40:32.602 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T11:40:34.204 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T11:40:34.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T11:40:34.506 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T11:40:34.641 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T11:40:34.684 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T11:40:34.833 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T11:40:35.182 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T11:40:35.341 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T11:40:35.603 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T11:40:35.743 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 94/122 2023-10-06T11:40:35.970 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 95/122 2023-10-06T11:40:36.147 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 96/122 2023-10-06T11:40:36.338 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T11:40:36.570 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 97/122 2023-10-06T11:40:36.773 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T11:40:37.807 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 98/122 2023-10-06T11:40:37.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T11:40:38.092 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 99/122 2023-10-06T11:40:38.472 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 100/122 2023-10-06T11:40:38.640 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T11:40:38.679 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 101/122 2023-10-06T11:40:38.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T11:40:38.866 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 102/122 2023-10-06T11:40:40.316 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T11:40:40.542 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 103/122 2023-10-06T11:40:40.618 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T11:40:40.757 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T11:40:40.815 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T11:40:40.848 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 104/122 2023-10-06T11:40:41.004 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T11:40:41.058 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1274.g68e53 105/122 2023-10-06T11:40:41.197 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T11:40:41.528 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T11:40:41.567 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 106/122 2023-10-06T11:40:41.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T11:40:41.870 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T11:40:42.049 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 111/122 2023-10-06T11:40:42.218 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T11:40:42.419 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T11:40:42.571 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T11:40:42.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T11:40:42.922 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T11:40:43.057 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T11:40:43.292 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T11:40:43.480 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T11:40:43.541 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T11:40:43.714 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T11:40:43.726 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T11:40:43.726 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T11:40:43.933 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T11:40:43.933 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:44.835 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T11:40:44.836 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T11:40:44.837 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T11:40:44.838 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T11:40:44.839 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T11:40:44.840 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T11:40:44.841 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T11:40:44.842 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T11:40:44.843 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T11:40:44.844 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T11:40:44.845 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T11:40:44.846 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T11:40:46.991 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 107/122 2023-10-06T11:40:47.033 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:47.033 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:42 ago on Fri 06 Oct 2023 11:39:05 AM UTC. 2023-10-06T11:40:47.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 108/122 2023-10-06T11:40:47.486 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T11:40:48.252 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T11:40:48.624 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T11:40:48.799 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 111/122 2023-10-06T11:40:48.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T11:40:49.133 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 113/122 2023-10-06T11:40:49.157 INFO:teuthology.orchestra.run.smithi177.stdout:Installed products updated. 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T11:40:49.299 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.300 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:49.301 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.5-12.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T11:40:49.302 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.303 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T11:40:49.304 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T11:40:49.305 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T11:40:49.306 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T11:40:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T11:40:49.308 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T11:40:49.309 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T11:40:49.310 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T11:40:49.311 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T11:40:49.311 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T11:40:49.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:40:49.311 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-10-06T11:40:49.462 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 115/122 2023-10-06T11:40:49.589 DEBUG:teuthology.parallel:result is None 2023-10-06T11:40:49.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 116/122 2023-10-06T11:40:49.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 117/122 2023-10-06T11:40:50.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T11:40:50.253 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T11:40:50.311 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T11:40:50.461 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T11:40:50.471 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T11:40:50.471 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T11:40:50.689 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T11:40:50.690 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:51.612 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 1/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 3/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 4/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1274.g68e53 5/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 6/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 7/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 8/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 9/122 2023-10-06T11:40:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 10/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 11/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 12/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 13/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_ 14/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 15/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 16/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_6 17/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86 18/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 19/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el 20/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8. 21/122 2023-10-06T11:40:51.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_6 22/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 23/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 24/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 25/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 26/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 27/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 28/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a. 29/122 2023-10-06T11:40:51.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noa 30/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.n 31/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e 32/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el 33/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 34/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.e 35/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 36/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 37/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 38/122 2023-10-06T11:40:51.623 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T11:40:51.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T11:40:51.625 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T11:40:51.626 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T11:40:51.627 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T11:40:51.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T11:40:51.629 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 119/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T11:40:51.630 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 121/122 2023-10-06T11:40:53.949 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T11:40:53.949 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:47 ago on Fri 06 Oct 2023 11:39:06 AM UTC. 2023-10-06T11:40:56.438 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.589 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:17.2.6-1274.g68e53a4a.el8.noarch 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T11:40:56.590 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.5-12.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T11:40:56.591 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T11:40:56.592 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T11:40:56.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T11:40:56.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T11:40:56.596 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:17.2.6-1274.g68e53a4a.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T11:40:56.597 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T11:40:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T11:40:56.598 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:40:56.598 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-10-06T11:40:56.916 DEBUG:teuthology.parallel:result is None 2023-10-06T11:40:56.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:40:57.147 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T11:40:57.169 INFO:teuthology.orchestra.run.smithi016.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T11:40:57.170 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T11:40:57.170 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T11:40:57.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564 2023-10-06T11:40:57.314 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T11:40:57.338 INFO:teuthology.orchestra.run.smithi177.stdout:17.2.6-1274.g68e53a4a.el8 2023-10-06T11:40:57.338 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1274.g68e53a4a.el8 2023-10-06T11:40:57.338 INFO:teuthology.task.install:The correct ceph version 17.2.6-1274.g68e53a4a is installed. 2023-10-06T11:40:57.338 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T11:40:57.339 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:57.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:40:57.379 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:57.379 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T11:40:57.421 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T11:40:57.421 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:57.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:40:57.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:40:57.512 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:57.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T11:40:57.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T11:40:57.603 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T11:40:57.603 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:57.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:40:57.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:40:57.694 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:57.694 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T11:40:57.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T11:40:57.782 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T11:40:57.783 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:57.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:40:57.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:40:57.873 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:57.873 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T11:40:57.897 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T11:40:57.962 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T11:40:58.026 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T11:40:58.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-10-06T11:40:58.029 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2023-10-06T11:40:58.052 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T11:40:58.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:40:58.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T11:40:58.118 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T11:40:58.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:40:58.159 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T11:40:58.189 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T11:40:58.190 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T11:40:58.190 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T11:40:58.190 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:58.190 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:40:58.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:58.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:58.376 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:58.376 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T11:40:58.390 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:58.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T11:40:58.532 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T11:40:58.532 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T11:40:58.532 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417', 'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412417', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4002, 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri3-testing-2023-10-05-0943-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564', 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBETzpEEBd5S9AGT46sVfXh0CmuWgCfE2KYq3uvkh21YMtkyX3fg8MjFIIC2e1shdrgADEeMosd/3XxXSmLp/6sY=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOHDdXoRoHZfnCPeoRS4nfiBlTkz0J5rsDHjszoMelrS3lw0SWeqNkRoWAXZuJv2SwvFbAftTTP/wEmA2FuoN0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564'}}, {'ceph': None}, {'ceph-fuse': None}, {'check-counter': {'counters': {'mgr': [{'min': 4, 'name': 'finisher-volumes.complete_latency.avgcount'}, {'expected_val': 0, 'name': 'finisher-volumes.queue_len'}]}}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestPerModuleFinsherThread']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:54:57', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T11:40:58.533 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:40:58.535 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T11:40:58.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:40:58.591 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T11:40:58.645 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:40:58.645 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:40:58.670 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:40:58.670 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 218724 Links: 1 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-10-06 11:40:51.165581118 +0000 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-10-06 11:38:21.419276128 +0000 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-10-06 11:38:21.419276128 +0000 2023-10-06T11:40:58.725 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-10-06T11:40:58.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:40:58.788 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-10-06T11:40:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-10-06T11:40:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000119165 s, 4.3 MB/s 2023-10-06T11:40:58.790 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:40:58.846 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-10-06T11:40:58.901 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 221299 Links: 1 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-10-06 11:40:51.165581118 +0000 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-10-06 11:38:21.919268250 +0000 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-10-06 11:38:21.919268250 +0000 2023-10-06T11:40:58.902 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-10-06T11:40:58.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:40:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-10-06T11:40:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-10-06T11:40:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.7076e-05 s, 5.3 MB/s 2023-10-06T11:40:58.967 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:40:59.023 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-10-06T11:40:59.078 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:40:59.078 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:59.078 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 222311 Links: 1 2023-10-06T11:40:59.078 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:59.079 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:59.079 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-10-06 11:40:51.165581118 +0000 2023-10-06T11:40:59.079 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-10-06 11:38:22.428260231 +0000 2023-10-06T11:40:59.079 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-10-06 11:38:22.428260231 +0000 2023-10-06T11:40:59.079 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-10-06T11:40:59.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:40:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-10-06T11:40:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-10-06T11:40:59.142 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000112065 s, 4.6 MB/s 2023-10-06T11:40:59.143 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:40:59.202 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 219011 Links: 1 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-10-06 11:40:51.165581118 +0000 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-10-06 11:38:22.937252212 +0000 2023-10-06T11:40:59.261 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-10-06 11:38:22.937252212 +0000 2023-10-06T11:40:59.262 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-10-06T11:40:59.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:40:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-10-06T11:40:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-10-06T11:40:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.4944e-05 s, 5.4 MB/s 2023-10-06T11:40:59.329 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:40:59.387 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:40:59.387 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:40:59.388 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T11:40:59.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T11:40:59.407 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2023-10-06T11:40:59.466 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 221342 Links: 1 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 11:40:44.410348297 +0000 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 11:38:06.207848001 +0000 2023-10-06T11:40:59.467 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 11:38:06.207848001 +0000 2023-10-06T11:40:59.468 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T11:40:59.468 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T11:40:59.532 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T11:40:59.532 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T11:40:59.533 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000277681 s, 1.8 MB/s 2023-10-06T11:40:59.533 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T11:40:59.592 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2023-10-06T11:40:59.652 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 221427 Links: 1 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 11:40:44.410348297 +0000 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 11:38:06.714834508 +0000 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 11:38:06.714834508 +0000 2023-10-06T11:40:59.653 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T11:40:59.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T11:40:59.718 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T11:40:59.718 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T11:40:59.718 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.00023786 s, 2.2 MB/s 2023-10-06T11:40:59.719 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T11:40:59.776 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2023-10-06T11:40:59.832 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T11:40:59.832 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:40:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 220859 Links: 1 2023-10-06T11:40:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:40:59.832 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:40:59.833 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 11:40:44.411348271 +0000 2023-10-06T11:40:59.833 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 11:38:07.190821841 +0000 2023-10-06T11:40:59.833 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 11:38:07.190821841 +0000 2023-10-06T11:40:59.833 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T11:40:59.833 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T11:40:59.896 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T11:40:59.896 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T11:40:59.896 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000222125 s, 2.3 MB/s 2023-10-06T11:40:59.897 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T11:40:59.953 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 216861 Links: 1 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 11:40:44.411348271 +0000 2023-10-06T11:41:00.008 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 11:38:07.665809200 +0000 2023-10-06T11:41:00.009 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 11:38:07.665809200 +0000 2023-10-06T11:41:00.009 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T11:41:00.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T11:41:00.072 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T11:41:00.072 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T11:41:00.072 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000207709 s, 2.5 MB/s 2023-10-06T11:41:00.072 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T11:41:00.127 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:00.127 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:41:00.127 INFO:tasks.ceph:Generating config... 2023-10-06T11:41:00.128 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T11:41:00.128 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T11:41:00.129 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T11:41:00.129 INFO:tasks.ceph:[client] pid file = 2023-10-06T11:41:00.129 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T11:41:00.129 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T11:41:00.129 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T11:41:00.129 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T11:41:00.129 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T11:41:00.129 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T11:41:00.130 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T11:41:00.131 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T11:41:00.131 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T11:41:00.131 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T11:41:00.131 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T11:41:00.131 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T11:41:00.131 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T11:41:00.132 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T11:41:00.133 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T11:41:00.133 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T11:41:00.133 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T11:41:00.133 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T11:41:00.133 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T11:41:00.133 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T11:41:00.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T11:41:00.182 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T11:41:00.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T11:41:00.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:41:00.292 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2023-10-06T11:41:00.292 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T11:41:00.293 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:00.293 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:41:00.347 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid b7fe6d1b-a342-44e2-9c53-ba3d26f05e41 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = octopus 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:fsid b7fe6d1b-a342-44e2-9c53-ba3d26f05e41 2023-10-06T11:41:00.430 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-10-06T11:41:00.429719+0000 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-10-06T11:41:00.429719+0000 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 15 (octopus) 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2023-10-06T11:41:00.431 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T11:41:00.432 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T11:41:00.486 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b7fe6d1b-a342-44e2-9c53-ba3d26f05e41... 2023-10-06T11:41:00.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:41:00.529 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T11:41:00.571 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-10-06T11:41:00.571 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-10-06T11:41:00.571 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:41:00.571 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.572 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-10-06T11:41:00.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:41:00.574 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-10-06T11:41:00.575 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = b7fe6d1b-a342-44e2-9c53-ba3d26f05e41 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.576 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:41:00.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-10-06T11:41:00.578 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:41:00.579 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-10-06T11:41:00.580 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug finisher = 20 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-10-06T11:41:00.581 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:41:00.582 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.583 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-10-06T11:41:00.584 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-10-06T11:41:00.585 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-10-06T11:41:00.586 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-10-06T11:41:00.587 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-10-06T11:41:00.587 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-10-06T11:41:00.588 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-10-06T11:41:00.588 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-10-06T11:41:00.589 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T11:41:00.590 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T11:41:00.591 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-10-06T11:41:00.592 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.16,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = b7fe6d1b-a342-44e2-9c53-ba3d26f05e41 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.593 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-10-06T11:41:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-10-06T11:41:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2023-10-06T11:41:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-10-06T11:41:00.594 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.595 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.596 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-10-06T11:41:00.597 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-10-06T11:41:00.598 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-10-06T11:41:00.599 INFO:teuthology.orchestra.run.smithi177.stdout: debug finisher = 20 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T11:41:00.600 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T11:41:00.601 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-10-06T11:41:00.602 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-10-06T11:41:00.603 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2023-10-06T11:41:00.604 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-10-06T11:41:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2023-10-06T11:41:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T11:41:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T11:41:00.605 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T11:41:00.605 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T11:41:00.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T11:41:00.683 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T11:41:00.683 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:00.683 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T11:41:00.698 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:00.698 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T11:41:00.753 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:41:00.753 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:00.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:41:00.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:41:00.829 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:00.830 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:00.885 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:41:00.886 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:00.886 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T11:41:00.886 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T11:41:00.920 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:00.920 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:00.975 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T11:41:00.975 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T11:41:00.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:41:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T11:41:01.026 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:41:01.082 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T11:41:01.085 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T11:41:01.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:41:01.134 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T11:41:01.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T11:41:01.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:41:01.250 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T11:41:01.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T11:41:01.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:41:01.326 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T11:41:01.328 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T11:41:01.352 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:41:01.439 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T11:41:01.441 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T11:41:01.465 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T11:41:01.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T11:41:01.504 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T11:41:01.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T11:41:01.595 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T11:41:01.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T11:41:01.647 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T11:41:01.659 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T11:41:01.738 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T11:41:01.750 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T11:41:01.751 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T11:41:01.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T11:41:01.776 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:01.776 INFO:tasks.ceph:role: osd.0 2023-10-06T11:41:01.776 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:41:01.776 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:01.841 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:01.842 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:01.864 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-10-06T11:41:01.865 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-10-06T11:41:01.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T11:41:01.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T11:41:02.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T11:41:02.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:02.079 INFO:tasks.ceph:role: osd.1 2023-10-06T11:41:02.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:41:02.079 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:41:02.143 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:02.143 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:02.144 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:02.166 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-10-06T11:41:02.168 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-10-06T11:41:02.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T11:41:02.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T11:41:02.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T11:41:02.380 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:02.380 INFO:tasks.ceph:role: osd.2 2023-10-06T11:41:02.380 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:41:02.380 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:41:02.444 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:02.444 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:02.445 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:02.468 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-10-06T11:41:02.469 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-10-06T11:41:02.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T11:41:02.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T11:41:02.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T11:41:02.682 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:02.682 INFO:tasks.ceph:role: osd.3 2023-10-06T11:41:02.682 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:41:02.683 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:41:02.747 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:02.747 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:02.748 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:02.770 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-10-06T11:41:02.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-10-06T11:41:02.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T11:41:02.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T11:41:02.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:03.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.006+0000 7f9b4b580540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T11:41:03.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.006+0000 7f9b4b580540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T11:41:03.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.006+0000 7f9b4b580540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:41:03.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.006+0000 7f9b4b580540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:41:03.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.006+0000 7f9b4b580540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T11:41:03.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.007+0000 7f9b4b580540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T11:41:03.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:03.528+0000 7f9b4b580540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:04.293+0000 7f9b4b580540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:05.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.056+0000 7f9b4b580540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:05.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T11:41:05.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.977+0000 7fc2b7260540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.977+0000 7fc2b7260540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.977+0000 7fc2b7260540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.977+0000 7fc2b7260540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.977+0000 7fc2b7260540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T11:41:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:05.978+0000 7fc2b7260540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T11:41:06.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:06.511+0000 7fc2b7260540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:07.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:07.271+0000 7fc2b7260540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.036+0000 7fc2b7260540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:08.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T11:41:08.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.947+0000 7fd5ec21e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T11:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.947+0000 7fd5ec21e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T11:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.947+0000 7fd5ec21e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.947+0000 7fd5ec21e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.947+0000 7fd5ec21e540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T11:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:08.948+0000 7fd5ec21e540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T11:41:09.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:09.470+0000 7fd5ec21e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:10.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:10.232+0000 7fd5ec21e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:11.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.003+0000 7fd5ec21e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:11.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T11:41:11.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.912+0000 7f0276a33540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T11:41:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.913+0000 7f0276a33540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T11:41:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.913+0000 7f0276a33540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:41:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.913+0000 7f0276a33540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:41:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.913+0000 7f0276a33540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T11:41:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:11.913+0000 7f0276a33540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T11:41:12.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:12.441+0000 7f0276a33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:13.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:13.203+0000 7f0276a33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:13.964+0000 7f0276a33540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:14.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T11:41:14.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T11:41:14.818 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:14.819 INFO:tasks.ceph:role: osd.4 2023-10-06T11:41:14.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:41:14.819 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T11:41:14.894 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:14.894 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:14.894 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:14.895 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:15.319 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T11:41:15.412 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T11:41:15.413 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T11:41:15.593 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T11:41:15.631 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T11:41:15.700 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:15.700 INFO:tasks.ceph:role: osd.5 2023-10-06T11:41:15.700 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:41:15.700 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T11:41:15.772 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:15.773 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:15.774 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:15.774 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:15.774 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:16.179 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T11:41:16.181 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T11:41:16.181 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T11:41:16.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T11:41:16.315 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T11:41:16.380 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:16.380 INFO:tasks.ceph:role: osd.6 2023-10-06T11:41:16.381 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:41:16.381 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T11:41:16.453 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:16.454 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:16.893 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T11:41:16.895 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T11:41:16.895 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T11:41:17.018 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T11:41:17.050 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T11:41:17.114 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T11:41:17.114 INFO:tasks.ceph:role: osd.7 2023-10-06T11:41:17.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:41:17.115 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T11:41:17.186 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T11:41:17.187 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T11:41:17.625 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T11:41:17.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T11:41:17.627 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T11:41:17.803 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T11:41:17.833 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:17.920 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.919+0000 7fd4cb791540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T11:41:17.921 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.920+0000 7fd4cb791540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T11:41:17.922 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.920+0000 7fd4cb791540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:41:17.922 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.920+0000 7fd4cb791540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:41:17.922 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.920+0000 7fd4cb791540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T11:41:17.931 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:17.929+0000 7fd4cb791540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T11:41:18.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:18.453+0000 7fd4cb791540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:19.219 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:19.218+0000 7fd4cb791540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:19.986 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:19.985+0000 7fd4cb791540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:20.787 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T11:41:20.814 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:20.899 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.897+0000 7faf3127e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T11:41:20.899 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.897+0000 7faf3127e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T11:41:20.899 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.898+0000 7faf3127e540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:41:20.899 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.898+0000 7faf3127e540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:41:20.899 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.898+0000 7faf3127e540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T11:41:20.900 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:20.898+0000 7faf3127e540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T11:41:21.422 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:21.421+0000 7faf3127e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:22.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:22.189+0000 7faf3127e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:22.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:22.951+0000 7faf3127e540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:23.745 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T11:41:23.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:23.907 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.905+0000 7fb96bf50540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T11:41:23.907 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.905+0000 7fb96bf50540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T11:41:23.907 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.905+0000 7fb96bf50540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:41:23.908 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.905+0000 7fb96bf50540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:41:23.908 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.905+0000 7fb96bf50540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T11:41:23.908 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:23.906+0000 7fb96bf50540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T11:41:24.431 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:24.430+0000 7fb96bf50540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:25.201 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:25.198+0000 7fb96bf50540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:25.962 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:25.961+0000 7fb96bf50540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:26.757 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T11:41:26.822 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:26.904 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T11:41:26.905 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T11:41:26.905 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:41:26.905 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:41:26.905 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T11:41:26.905 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:26.903+0000 7ff856ae0540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T11:41:27.427 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:27.425+0000 7ff856ae0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:28.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:28.191+0000 7ff856ae0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:28.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:28.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:28.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:28.957+0000 7ff856ae0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T11:41:29.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T11:41:29.773 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T11:41:29.773 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:29.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T11:41:29.801 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:29.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T11:41:29.868 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:29.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T11:41:29.935 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:29.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T11:41:30.000 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:30.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T11:41:30.065 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:30.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T11:41:30.131 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:30.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T11:41:30.196 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.196 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T11:41:30.221 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T11:41:30.286 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.286 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T11:41:30.350 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.350 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T11:41:30.413 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.414 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T11:41:30.478 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.478 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T11:41:30.542 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.542 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T11:41:30.606 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:30.607 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T11:41:30.622 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:30.622 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T11:41:30.678 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.678 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T11:41:30.693 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:30.693 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T11:41:30.748 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T11:41:30.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:41:30.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T11:41:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-10-06T11:41:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9l02ZvARAAjJeuClekdfh0vdVcSsCD4Q== 2023-10-06T11:41:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-10-06T11:41:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9ltSYCCBAA0LbHS7AfdXp0X5kyPd1QVQ== 2023-10-06T11:41:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9lxKLhDhAA/nrhDdOuamujt6Bn65YDHg== 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBP8h9lNwF6ABAAQSocwfWfDqRCUFZNXVYMUA== 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBR8h9lylNcOhAA6MVxZoYsfNNH4Ba1jtGcTg== 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBU8h9lpWeCOBAAZ8qBWvfpx/Z0MrN1m4jMIA== 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-10-06T11:41:30.792 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBX8h9lyZVuNhAALXqD6pr+4wTEqa5Yps7K7Q== 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9l8rTmBBAA3basAsynbmSDIeHx2VWeaA== 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9loZRoExAAcVM5vfC8B3rscVnEOsVkqg== 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9lwkwrGhAAp+yW5zQWHG9RflVm/EKYyA== 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBd8h9l3dfRNhAASUc1GOR8xGzcsAV55z8YAQ== 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-10-06T11:41:30.793 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBg8h9l8UaLNRAAq/hqizhAVtfkujBKnTznDA== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBj8h9lb3kCNhAAI6n2UoNJ9lbq+GnReN1neA== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBm8h9liT3eNRAASOGudkgH0uyd405p/NIqaQ== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9lvHkHHhAAjOHhMp7Y737Y8jx3RQR1uQ== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9llgR7IxAA6k0an30dLNvVRta4W6S+QA== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9lqCCMJhAATgZVBarXJ3fS0xLhA7UidA== 2023-10-06T11:41:30.794 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2023-10-06T11:41:30.795 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBN8h9lS972KxAA8CoEqWxe/erJBuYO+g3lcQ== 2023-10-06T11:41:30.810 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2023-10-06T11:41:30.810 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9l02ZvARAAjJeuClekdfh0vdVcSsCD4Q== 2023-10-06T11:41:30.810 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9ltSYCCBAA0LbHS7AfdXp0X5kyPd1QVQ== 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9lxKLhDhAA/nrhDdOuamujt6Bn65YDHg== 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBP8h9lNwF6ABAAQSocwfWfDqRCUFZNXVYMUA== 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBR8h9lylNcOhAA6MVxZoYsfNNH4Ba1jtGcTg== 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBU8h9lpWeCOBAAZ8qBWvfpx/Z0MrN1m4jMIA== 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2023-10-06T11:41:30.811 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBX8h9lyZVuNhAALXqD6pr+4wTEqa5Yps7K7Q== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9l8rTmBBAA3basAsynbmSDIeHx2VWeaA== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9loZRoExAAcVM5vfC8B3rscVnEOsVkqg== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9lwkwrGhAAp+yW5zQWHG9RflVm/EKYyA== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBd8h9l3dfRNhAASUc1GOR8xGzcsAV55z8YAQ== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBg8h9l8UaLNRAAq/hqizhAVtfkujBKnTznDA== 2023-10-06T11:41:30.812 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBj8h9lb3kCNhAAI6n2UoNJ9lbq+GnReN1neA== 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBm8h9liT3eNRAASOGudkgH0uyd405p/NIqaQ== 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9lvHkHHhAAjOHhMp7Y737Y8jx3RQR1uQ== 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9llgR7IxAA6k0an30dLNvVRta4W6S+QA== 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9lqCCMJhAATgZVBarXJ3fS0xLhA7UidA== 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2023-10-06T11:41:30.813 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBN8h9lS972KxAA8CoEqWxe/erJBuYO+g3lcQ== 2023-10-06T11:41:30.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:41:30.839 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:41:30.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:30.904 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:30.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:30.957 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:31.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.009 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.107 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.157 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:41:31.206 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T11:41:31.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:31.254 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:31.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:31.306 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T11:41:31.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.408 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.458 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.507 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T11:41:31.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.556 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.608 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.658 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.660 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.712 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T11:41:31.762 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T11:41:31.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T11:41:31.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:41:31.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T11:41:31.906 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T11:41:31.930 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:41:32.022 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T11:41:32.046 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T11:41:32.111 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T11:41:32.206 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T11:41:32.230 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:32.232 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T11:41:32.284 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T11:41:32.285 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T11:41:32.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T11:41:32.287 INFO:tasks.ceph.mon.a:Started 2023-10-06T11:41:32.287 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T11:41:32.287 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T11:41:32.327 INFO:tasks.ceph.mon.b:Started 2023-10-06T11:41:32.327 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T11:41:32.327 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T11:41:32.329 INFO:tasks.ceph.mon.c:Started 2023-10-06T11:41:32.329 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T11:41:32.329 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T11:41:32.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T11:41:32.332 INFO:tasks.ceph.mgr.y:Started 2023-10-06T11:41:32.332 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T11:41:32.332 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T11:41:32.333 INFO:tasks.ceph.mgr.x:Started 2023-10-06T11:41:32.334 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T11:41:32.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T11:41:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.453+0000 7f2013dbe700 1 Processor -- start 2023-10-06T11:41:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.453+0000 7f2013dbe700 1 -- start start 2023-10-06T11:41:32.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2013dbe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2013dbe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2013dbe700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c0cead0 con 0x7f200c159b10 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2013dbe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c0affc0 con 0x7f200c0cda10 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2013dbe700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200c0cdec0 con 0x7f200c159740 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46410/0 (socket says 172.21.15.16:46410) 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 learned_addr learned my addr 172.21.15.16:0/2111109198 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:32.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:41:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:41:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:32.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.454+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:41:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:32.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:32.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:32.655+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:41:33.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:33.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:41:33.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:33.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.056+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:41:33.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011b5a700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:33.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:33.857+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 msgr2=0x7f200c0b3990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.177:3300/0 conn(0x7f200c159740 0x7f200c0b3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 -- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 msgr2=0x7f200c0b3ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 --2- 172.21.15.16:0/2111109198 >> v2:172.21.15.16:3300/0 conn(0x7f200c0cda10 0x7f200c0b3ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 -- 172.21.15.16:0/2111109198 >> v1:172.21.15.177:6789/0 conn(0x7f200c159b10 legacy=0x7f200c15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 -- 172.21.15.16:0/2111109198 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200c0ce1a0 con 0x7f1ff80036e0 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f200b7fe700 1 -- 172.21.15.16:0/2111109198 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0affc0 con 0x7f1ff8007320 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:35.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.454+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T11:41:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.655+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.655+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:35.655+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T11:41:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.056+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.056+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:36.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.056+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T11:41:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.857+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.858+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T11:41:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.858+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T11:41:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.858+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T11:41:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:36.858+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T11:41:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279666743 0 0) 0x7f200c0ce1a0 con 0x7f1ff80036e0 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff4003570 con 0x7f1ff80036e0 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597300353 0 0) 0x7f1ff4003570 con 0x7f1ff80036e0 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 msgr2=0x7f1ff80097b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff8007320 0x7f1ff80097b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff800af80 con 0x7f1ff80036e0 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f1ffc00d300 con 0x7f1ff80036e0 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 >> v1:172.21.15.16:6789/0 conn(0x7f1ff80036e0 legacy=0x7f1ff8005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f201235b700 1 -- 172.21.15.16:0/2111109198 reap_dead start 2023-10-06T11:41:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f1ff4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f1ff4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f1ff400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c0affc0 con 0x7f1ff4004810 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff400c250 con 0x7f1ff4003a60 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff400c410 con 0x7f1ff40052e0 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f1ff4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.670+0000 7f2011b5a700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f1ff4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f201235b700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f1ff400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 msgr2=0x7f1ff400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f1ff400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 msgr2=0x7f1ff4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f1ff4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 -- 172.21.15.16:0/2111109198 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4003570 con 0x7f1ff4004810 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2011359700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f1ff4004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2000009060 tx=0x7f2000008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c59fb8c1df51a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200001e070 con 0x7f1ff4004810 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200000c040 con 0x7f1ff4004810 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.671+0000 7f2010b58700 1 -- 172.21.15.16:0/2111109198 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000010b50 con 0x7f1ff4004810 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 -- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 msgr2=0x7f1ff4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f1ff4004c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2000009060 tx=0x7f2000008040 comp rx=0 tx=0).stop 2023-10-06T11:41:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 -- 172.21.15.16:0/2111109198 shutdown_connections 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f1ff4004c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f1ff400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 --2- 172.21.15.16:0/2111109198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f1ff4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 -- 172.21.15.16:0/2111109198 >> 172.21.15.16:0/2111109198 conn(0x7f200c099a40 msgr2=0x7f200c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 -- 172.21.15.16:0/2111109198 shutdown_connections 2023-10-06T11:41:37.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.672+0000 7f2013dbe700 1 -- 172.21.15.16:0/2111109198 wait complete. 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 Processor -- start 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 -- start start 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f200c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f200c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c09e340 con 0x7f1ff4004810 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1544a0 con 0x7f1ff4003a60 2023-10-06T11:41:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.674+0000 7f2013dbe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200c15cf60 con 0x7f1ff40052e0 2023-10-06T11:41:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46480/0 (socket says 172.21.15.16:46480) 2023-10-06T11:41:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 -- 172.21.15.16:0/975925965 learned_addr learned my addr 172.21.15.16:0/975925965 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011b5a700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f200c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f201235b700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f200c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 -- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 msgr2=0x7f200c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f200c15d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 -- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 msgr2=0x7f200c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f200c15ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 -- 172.21.15.16:0/975925965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c09b770 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2011359700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2000022e90 tx=0x7f2000027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000010b50 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2000000bc0 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.675+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c1601e0 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.676+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000035630 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.676+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c09b400 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.676+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f2000029070 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.676+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f200003f740 con 0x7f1ff4004810 2023-10-06T11:41:37.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.677+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c09b900 con 0x7f1ff4004810 2023-10-06T11:41:37.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.679+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f200003a020 con 0x7f1ff4004810 2023-10-06T11:41:37.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.758+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f200c04c380 con 0x7f1ff4004810 2023-10-06T11:41:37.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.760+0000 7f2009ffb700 1 -- 172.21.15.16:0/975925965 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2000031070 con 0x7f1ff4004810 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 msgr2=0x7f200c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2000022e90 tx=0x7f2000027000 comp rx=0 tx=0).stop 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 shutdown_connections 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ff40052e0 0x7f200c15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ff4003a60 0x7f200c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 --2- 172.21.15.16:0/975925965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff4004810 0x7f200c15d480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 >> 172.21.15.16:0/975925965 conn(0x7f200c099a40 msgr2=0x7f200c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 shutdown_connections 2023-10-06T11:41:37.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:37.761+0000 7f2013dbe700 1 -- 172.21.15.16:0/975925965 wait complete. 2023-10-06T11:41:37.765 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-10-06T11:41:37.776 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T11:41:37.776 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:37.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T11:41:37.802 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:37.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T11:41:37.868 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:37.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T11:41:37.933 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-10-06T11:41:37.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T11:41:37.998 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:37.998 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T11:41:38.027 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:38.027 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T11:41:38.093 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:38.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T11:41:38.160 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T11:41:38.161 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T11:41:38.227 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new af0d6ca8-3aba-4727-abbc-b852e772b0b9 0 2023-10-06T11:41:38.383 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.381+0000 7f650ff81700 1 Processor -- start 2023-10-06T11:41:38.384 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.381+0000 7f650ff81700 1 -- start start 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650ff81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 0x7f6508159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650ff81700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 0x7f650806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650ff81700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650806cde0 con 0x7f650806d270 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650ff81700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080c2550 con 0x7f650806d6e0 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650ff81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650806c4b0 con 0x7f6508159740 2023-10-06T11:41:38.386 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650dd1d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 0x7f6508159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650d51c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f650806d270 0x7f65080c1ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:56384/0 (socket says 172.21.15.177:56384) 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.382+0000 7f650d51c700 1 -- 172.21.15.177:0/1070742026 learned_addr learned my addr 172.21.15.177:0/1070742026 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650cd1b700 1 -- 172.21.15.177:0/1070742026 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618318743 0 0) 0x7f650806cde0 con 0x7f650806d270 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650dd1d700 1 -- 172.21.15.177:0/1070742026 >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 msgr2=0x7f650806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650e51e700 1 --2- 172.21.15.177:0/1070742026 >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 0x7f650806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650dd1d700 1 --2- 172.21.15.177:0/1070742026 >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 0x7f650806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650dd1d700 1 -- 172.21.15.177:0/1070742026 >> v1:172.21.15.177:6789/0 conn(0x7f650806d270 legacy=0x7f65080c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650dd1d700 1 -- 172.21.15.177:0/1070742026 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650806c790 con 0x7f6508159740 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650e51e700 1 --2- 172.21.15.177:0/1070742026 >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 0x7f650806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650dd1d700 1 --2- 172.21.15.177:0/1070742026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 0x7f6508159b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65040098e0 tx=0x7f65040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4539f464acb45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650cd1b700 1 -- 172.21.15.177:0/1070742026 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504009e80 con 0x7f6508159740 2023-10-06T11:41:38.387 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650cd1b700 1 -- 172.21.15.177:0/1070742026 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650400b860 con 0x7f6508159740 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650cd1b700 1 -- 172.21.15.177:0/1070742026 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400bbe0 con 0x7f6508159740 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 -- 172.21.15.177:0/1070742026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 msgr2=0x7f6508159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 --2- 172.21.15.177:0/1070742026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 0x7f6508159b10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f65040098e0 tx=0x7f65040091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 -- 172.21.15.177:0/1070742026 shutdown_connections 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 --2- 172.21.15.177:0/1070742026 >> v2:172.21.15.16:3300/0 conn(0x7f650806d6e0 0x7f650806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 --2- 172.21.15.177:0/1070742026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6508159740 0x7f6508159b10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.383+0000 7f650ff81700 1 -- 172.21.15.177:0/1070742026 >> 172.21.15.177:0/1070742026 conn(0x7f65080c95e0 msgr2=0x7f65080c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.384+0000 7f650ff81700 1 -- 172.21.15.177:0/1070742026 shutdown_connections 2023-10-06T11:41:38.388 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.384+0000 7f650ff81700 1 -- 172.21.15.177:0/1070742026 wait complete. 2023-10-06T11:41:38.390 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 Processor -- start 2023-10-06T11:41:38.390 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 -- start start 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 0x7f650815b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 0x7f65080b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650815ace0 con 0x7f6508159740 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b23d0 con 0x7f650806d270 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650ff81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080b1470 con 0x7f650806d6e0 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650dd1d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 0x7f650815b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650d51c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650d51c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33886/0 (socket says 172.21.15.177:33886) 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650dd1d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 0x7f650815b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:60942/0 (socket says 172.21.15.177:60942) 2023-10-06T11:41:38.391 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.385+0000 7f650d51c700 1 -- 172.21.15.177:0/2072360665 learned_addr learned my addr 172.21.15.177:0/2072360665 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650e51e700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 0x7f65080b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 -- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 msgr2=0x7f650815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 0x7f650815b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 -- 172.21.15.177:0/2072360665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 msgr2=0x7f65080b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 0x7f65080b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 -- 172.21.15.177:0/2072360665 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504009420 con 0x7f650806d6e0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650e51e700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 0x7f65080b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650d51c700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c5b0 tx=0x7f64f8001070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8005b30 con 0x7f650806d6e0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8005c90 con 0x7f650806d6e0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650815e050 con 0x7f650806d6e0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f8007300 con 0x7f650806d6e0 2023-10-06T11:41:38.392 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f64f8012680 con 0x7f650806d6e0 2023-10-06T11:41:38.393 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650815b1a0 con 0x7f650806d6e0 2023-10-06T11:41:38.393 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.386+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f64f8005e00 con 0x7f650806d6e0 2023-10-06T11:41:38.393 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.387+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65080b1630 con 0x7f650806d6e0 2023-10-06T11:41:38.393 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.389+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f64f8017070 con 0x7f650806d6e0 2023-10-06T11:41:38.475 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.473+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "af0d6ca8-3aba-4727-abbc-b852e772b0b9", "id": 0} v 0) v1 -- 0x7f650804c380 con 0x7f650806d6e0 2023-10-06T11:41:38.478 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.476+0000 7f64fdffb700 1 -- 172.21.15.177:0/2072360665 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af0d6ca8-3aba-4727-abbc-b852e772b0b9", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f64f80076f0 con 0x7f650806d6e0 2023-10-06T11:41:38.478 INFO:teuthology.orchestra.run.smithi177.stdout:0 2023-10-06T11:41:38.479 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 msgr2=0x7f65080b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.479 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c5b0 tx=0x7f64f8001070 comp rx=0 tx=0).stop 2023-10-06T11:41:38.479 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 shutdown_connections 2023-10-06T11:41:38.479 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6508159740 0x7f65080b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.480 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f650806d270 0x7f650815b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.480 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 --2- 172.21.15.177:0/2072360665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f650806d6e0 0x7f65080b1950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.480 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 >> 172.21.15.177:0/2072360665 conn(0x7f65080c95e0 msgr2=0x7f65080cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.480 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 shutdown_connections 2023-10-06T11:41:38.480 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.477+0000 7f650ff81700 1 -- 172.21.15.177:0/2072360665 wait complete. 2023-10-06T11:41:38.490 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new d38f9116-9742-4c46-bd6e-20844ce41a41 1 2023-10-06T11:41:38.606 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.604+0000 7f2973dde700 1 Processor -- start 2023-10-06T11:41:38.607 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.604+0000 7f2973dde700 1 -- start start 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2973dde700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 0x7f296c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2973dde700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f296c15dc10 0x7f296c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2973dde700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c13c9f0 con 0x7f296c0cb660 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2973dde700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c15e270 con 0x7f296c15dc10 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2973dde700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c15e050 con 0x7f296c0cb290 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f297237b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f296c15dc10 0x7f296c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.609 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f297237b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f296c15dc10 0x7f296c0cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:60966/0 (socket says 172.21.15.177:60966) 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2971379700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f296c0cb660 0x7f296c15d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:56400/0 (socket says 172.21.15.177:56400) 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f2971b7a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 0x7f296c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f297237b700 1 -- 172.21.15.177:0/2405056628 learned_addr learned my addr 172.21.15.177:0/2405056628 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.605+0000 7f297237b700 1 -- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 msgr2=0x7f296c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f297237b700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 0x7f296c0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f297237b700 1 -- 172.21.15.177:0/2405056628 >> v1:172.21.15.177:6789/0 conn(0x7f296c0cb660 legacy=0x7f296c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f297237b700 1 -- 172.21.15.177:0/2405056628 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c06dff0 con 0x7f296c15dc10 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2970b78700 1 -- 172.21.15.177:0/2405056628 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004335022 0 0) 0x7f2968001110 con 0x7f296c0cb660 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2971b7a700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 0x7f296c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f297237b700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c0cc600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2964009f20 tx=0x7f2964009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e9a66789b124d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.610 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2970b78700 1 -- 172.21.15.177:0/2405056628 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400e070 con 0x7f296c15dc10 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2970b78700 1 -- 172.21.15.177:0/2405056628 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296400b9d0 con 0x7f296c15dc10 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2970b78700 1 -- 172.21.15.177:0/2405056628 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640092e0 con 0x7f296c15dc10 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 -- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 msgr2=0x7f296c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c0cc600 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2964009f20 tx=0x7f2964009840 comp rx=0 tx=0).stop 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 -- 172.21.15.177:0/2405056628 shutdown_connections 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c0cc600 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 --2- 172.21.15.177:0/2405056628 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb290 0x7f296c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.606+0000 7f2973dde700 1 -- 172.21.15.177:0/2405056628 >> 172.21.15.177:0/2405056628 conn(0x7f296c067c90 msgr2=0x7f296c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.607+0000 7f2973dde700 1 -- 172.21.15.177:0/2405056628 shutdown_connections 2023-10-06T11:41:38.611 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.607+0000 7f2973dde700 1 -- 172.21.15.177:0/2405056628 wait complete. 2023-10-06T11:41:38.613 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 Processor -- start 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 -- start start 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cb290 0x7f296c16a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 0x7f296c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c16b110 con 0x7f296c0cb290 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1596b0 con 0x7f296c15dc10 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2973dde700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c159990 con 0x7f296c0cb660 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f297237b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f2971379700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 0x7f296c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f297237b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:60972/0 (socket says 172.21.15.177:60972) 2023-10-06T11:41:38.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.608+0000 7f297237b700 1 -- 172.21.15.177:0/3835469210 learned_addr learned my addr 172.21.15.177:0/3835469210 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971b7a700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cb290 0x7f296c16a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 -- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 msgr2=0x7f296c15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 -- 172.21.15.177:0/3835469210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cb290 msgr2=0x7f296c16a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cb290 0x7f296c16a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 -- 172.21.15.177:0/3835469210 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009420 con 0x7f296c0cb660 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f297237b700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2971379700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 0x7f296c13be60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f296800c7e0 tx=0x7f2968007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968005a70 con 0x7f296c0cb660 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2968005bf0 con 0x7f296c0cb660 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964009ae0 con 0x7f296c0cb660 2023-10-06T11:41:38.615 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680070d0 con 0x7f296c0cb660 2023-10-06T11:41:38.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c159f50 con 0x7f296c0cb660 2023-10-06T11:41:38.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f296801c440 con 0x7f296c0cb660 2023-10-06T11:41:38.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.609+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f29680127c0 con 0x7f296c0cb660 2023-10-06T11:41:38.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.610+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c15a170 con 0x7f296c0cb660 2023-10-06T11:41:38.616 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.612+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2968017070 con 0x7f296c0cb660 2023-10-06T11:41:38.699 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.697+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d38f9116-9742-4c46-bd6e-20844ce41a41", "id": 1} v 0) v1 -- 0x7f296c04c380 con 0x7f296c0cb660 2023-10-06T11:41:38.702 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.700+0000 7f2961ffb700 1 -- 172.21.15.177:0/3835469210 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d38f9116-9742-4c46-bd6e-20844ce41a41", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f296c13c9f0 con 0x7f296c0cb660 2023-10-06T11:41:38.702 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T11:41:38.703 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 msgr2=0x7f296c13be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.703 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 0x7f296c13be60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f296800c7e0 tx=0x7f2968007b90 comp rx=0 tx=0).stop 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 shutdown_connections 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f296c0cb290 0x7f296c16a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f296c15dc10 0x7f296c15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 --2- 172.21.15.177:0/3835469210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f296c0cb660 0x7f296c13be60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 >> 172.21.15.177:0/3835469210 conn(0x7f296c067c90 msgr2=0x7f296c15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 shutdown_connections 2023-10-06T11:41:38.704 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.701+0000 7f2973dde700 1 -- 172.21.15.177:0/3835469210 wait complete. 2023-10-06T11:41:38.715 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 0afa886b-62c7-409b-907c-81d69b99b2a8 2 2023-10-06T11:41:38.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.829+0000 7fef4e89c700 1 Processor -- start 2023-10-06T11:41:38.832 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.829+0000 7fef4e89c700 1 -- start start 2023-10-06T11:41:38.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef4e89c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fef48159740 0x7fef480b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef4e89c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef48159b10 0x7fef480cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef4e89c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480cead0 con 0x7fef480cda10 2023-10-06T11:41:38.834 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef4e89c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480affc0 con 0x7fef48159b10 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef4e89c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480cdec0 con 0x7fef48159740 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef47fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fef48159740 0x7fef480b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef47fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fef48159740 0x7fef480b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:60980/0 (socket says 172.21.15.177:60980) 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.830+0000 7fef47fff700 1 -- 172.21.15.177:0/2393542539 learned_addr learned my addr 172.21.15.177:0/2393542539 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef47fff700 1 -- 172.21.15.177:0/2393542539 >> v2:172.21.15.16:3300/0 conn(0x7fef48159b10 msgr2=0x7fef480cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef47fff700 1 --2- 172.21.15.177:0/2393542539 >> v2:172.21.15.16:3300/0 conn(0x7fef48159b10 0x7fef480cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef47fff700 1 -- 172.21.15.177:0/2393542539 >> v1:172.21.15.16:6789/0 conn(0x7fef480cda10 legacy=0x7fef480b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef47fff700 1 -- 172.21.15.177:0/2393542539 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480ce1a0 con 0x7fef48159740 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef47fff700 1 --2- 172.21.15.177:0/2393542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159740 0x7fef480b3990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef48089030 tx=0x7fef3c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95f0483b4c9a57e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef46ffd700 1 -- 172.21.15.177:0/2393542539 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00b820 con 0x7fef48159740 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef46ffd700 1 -- 172.21.15.177:0/2393542539 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00b980 con 0x7fef48159740 2023-10-06T11:41:38.835 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef46ffd700 1 -- 172.21.15.177:0/2393542539 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00bd20 con 0x7fef48159740 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef4e89c700 1 -- 172.21.15.177:0/2393542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159740 msgr2=0x7fef480b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.831+0000 7fef4e89c700 1 --2- 172.21.15.177:0/2393542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159740 0x7fef480b3990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fef48089030 tx=0x7fef3c009250 comp rx=0 tx=0).stop 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 -- 172.21.15.177:0/2393542539 shutdown_connections 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 --2- 172.21.15.177:0/2393542539 >> v2:172.21.15.16:3300/0 conn(0x7fef48159b10 0x7fef480cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 --2- 172.21.15.177:0/2393542539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159740 0x7fef480b3990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 -- 172.21.15.177:0/2393542539 >> 172.21.15.177:0/2393542539 conn(0x7fef48099a40 msgr2=0x7fef4806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 -- 172.21.15.177:0/2393542539 shutdown_connections 2023-10-06T11:41:38.836 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.832+0000 7fef4e89c700 1 -- 172.21.15.177:0/2393542539 wait complete. 2023-10-06T11:41:38.838 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.833+0000 7fef4e89c700 1 Processor -- start 2023-10-06T11:41:38.838 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.833+0000 7fef4e89c700 1 -- start start 2023-10-06T11:41:38.838 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef480cda10 0x7fef48155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef48159740 0x7fef480b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4816d800 con 0x7fef480cda10 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4816dae0 con 0x7fef48159b10 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4816ddc0 con 0x7fef48159740 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef477fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef48159740 0x7fef480b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:60990/0 (socket says 172.21.15.177:60990) 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 -- 172.21.15.177:0/1744958596 learned_addr learned my addr 172.21.15.177:0/1744958596 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef47fff700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef480cda10 0x7fef48155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 -- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef48159740 msgr2=0x7fef480b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.839 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef48159740 0x7fef480b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 -- 172.21.15.177:0/1744958596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef480cda10 msgr2=0x7fef48155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef480cda10 0x7fef48155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 -- 172.21.15.177:0/1744958596 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c0094f0 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4ce39700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef4000bc30 tx=0x7fef40009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400092e0 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4000c630 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4816e0a0 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000ca10 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.834+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef48155aa0 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.835+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fef40012070 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.835+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fef40015da0 con 0x7fef48159b10 2023-10-06T11:41:38.840 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.835+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4809a420 con 0x7fef48159b10 2023-10-06T11:41:38.841 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.839+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fef4809a420 con 0x7fef48159b10 2023-10-06T11:41:38.924 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.922+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0afa886b-62c7-409b-907c-81d69b99b2a8", "id": 2} v 0) v1 -- 0x7fef4804c380 con 0x7fef48159b10 2023-10-06T11:41:38.927 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.925+0000 7fef2ffff700 1 -- 172.21.15.177:0/1744958596 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0afa886b-62c7-409b-907c-81d69b99b2a8", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef40015470 con 0x7fef48159b10 2023-10-06T11:41:38.927 INFO:teuthology.orchestra.run.smithi177.stdout:2 2023-10-06T11:41:38.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 msgr2=0x7fef4809d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:38.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef4000bc30 tx=0x7fef40009840 comp rx=0 tx=0).stop 2023-10-06T11:41:38.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 shutdown_connections 2023-10-06T11:41:38.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef480cda10 0x7fef48155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.928 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fef48159b10 0x7fef4809d930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.929 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 --2- 172.21.15.177:0/1744958596 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fef48159740 0x7fef480b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:38.929 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 >> 172.21.15.177:0/1744958596 conn(0x7fef48099a40 msgr2=0x7fef481565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:38.929 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 shutdown_connections 2023-10-06T11:41:38.929 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:38.926+0000 7fef4e89c700 1 -- 172.21.15.177:0/1744958596 wait complete. 2023-10-06T11:41:38.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new e28cf380-7e4a-4a73-870d-f37e2436099c 3 2023-10-06T11:41:39.086 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.084+0000 7f0c2219e700 1 Processor -- start 2023-10-06T11:41:39.086 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.084+0000 7f0c2219e700 1 -- start start 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c2219e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c1c159740 0x7f0c1c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c2219e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b1790 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c2219e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b0e60 con 0x7f0c1c0b4790 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c2219e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b1140 con 0x7f0c1c159740 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1bfff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0c1c0b4790 0x7f0c1c0aa820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:56416/0 (socket says 172.21.15.177:56416) 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1bfff700 1 -- 172.21.15.177:0/376094317 learned_addr learned my addr 172.21.15.177:0/376094317 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1b7fe700 1 --2- 172.21.15.177:0/376094317 >> v2:172.21.15.16:3300/0 conn(0x7f0c1c159740 0x7f0c1c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794878770 0 0) 0x7f0c1c0b0e60 con 0x7f0c1c0b4790 2023-10-06T11:41:39.089 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c04003570 con 0x7f0c1c0b4790 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/376094317 >> v1:172.21.15.177:6789/0 conn(0x7f0c1c0b4790 legacy=0x7f0c1c0aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/376094317 >> v1:172.21.15.16:6789/0 conn(0x7f0c1c0b09d0 legacy=0x7f0c1c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168309452 0 0) 0x7f0c1c0b1790 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/376094317 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0b1420 con 0x7f0c1c159740 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.085+0000 7f0c1b7fe700 1 --2- 172.21.15.177:0/376094317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c159740 0x7f0c1c159b10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0098e0 tx=0x7f0c0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc19ec1ce36e2e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00b6d0 con 0x7f0c1c159740 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00b830 con 0x7f0c1c159740 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c1a7fc700 1 -- 172.21.15.177:0/376094317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00bbd0 con 0x7f0c1c159740 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 -- 172.21.15.177:0/376094317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c159740 msgr2=0x7f0c1c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 --2- 172.21.15.177:0/376094317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c159740 0x7f0c1c159b10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0098e0 tx=0x7f0c0c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.091 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 -- 172.21.15.177:0/376094317 shutdown_connections 2023-10-06T11:41:39.091 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 --2- 172.21.15.177:0/376094317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c159740 0x7f0c1c159b10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.091 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 -- 172.21.15.177:0/376094317 >> 172.21.15.177:0/376094317 conn(0x7f0c1c099a40 msgr2=0x7f0c1c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.091 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 -- 172.21.15.177:0/376094317 shutdown_connections 2023-10-06T11:41:39.091 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.086+0000 7f0c2219e700 1 -- 172.21.15.177:0/376094317 wait complete. 2023-10-06T11:41:39.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.087+0000 7f0c2219e700 1 Processor -- start 2023-10-06T11:41:39.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.087+0000 7f0c2219e700 1 -- start start 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c0b4790 0x7f0c1c15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c1c159740 0x7f0c1c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c09b610 con 0x7f0c1c0b4790 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c15f4d0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c2219e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c09afc0 con 0x7f0c1c159740 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c1b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c1b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32772/0 (socket says 172.21.15.177:32772) 2023-10-06T11:41:39.094 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/83481347 learned_addr learned my addr 172.21.15.177:0/83481347 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c1bfff700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c1c159740 0x7f0c1c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.088+0000 7f0c1affd700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c0b4790 0x7f0c1c15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c1c159740 msgr2=0x7f0c1c15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c1c159740 0x7f0c1c15eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/83481347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c0b4790 msgr2=0x7f0c1c15e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c0b4790 0x7f0c1c15e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 -- 172.21.15.177:0/83481347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c009420 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c1b7fe700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0093f0 tx=0x7f0c0c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c01f070 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c000bc0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c09b2a0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00bbb0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c09abe0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0c0c023070 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.089+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f0c0c0118c0 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.090+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c09ae00 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.092+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0c0c02a070 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.176 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.174+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e28cf380-7e4a-4a73-870d-f37e2436099c", "id": 3} v 0) v1 -- 0x7f0c1c04c380 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.179 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.177+0000 7f0c037fe700 1 -- 172.21.15.177:0/83481347 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e28cf380-7e4a-4a73-870d-f37e2436099c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c0c015020 con 0x7f0c1c0b09d0 2023-10-06T11:41:39.180 INFO:teuthology.orchestra.run.smithi177.stdout:3 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 msgr2=0x7f0c1c15c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0093f0 tx=0x7f0c0c017000 comp rx=0 tx=0).stop 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 shutdown_connections 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c1c159740 0x7f0c1c15eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c1c0b09d0 0x7f0c1c15c890 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 --2- 172.21.15.177:0/83481347 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1c0b4790 0x7f0c1c15e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 >> 172.21.15.177:0/83481347 conn(0x7f0c1c099a40 msgr2=0x7f0c1c0b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 shutdown_connections 2023-10-06T11:41:39.181 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.178+0000 7f0c2219e700 1 -- 172.21.15.177:0/83481347 wait complete. 2023-10-06T11:41:39.192 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new a175cb92-fb78-4cbb-91f5-5df435916564 4 2023-10-06T11:41:39.308 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 Processor -- start 2023-10-06T11:41:39.308 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 -- start start 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f00cead0 con 0x7f56f00cda10 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00affc0 con 0x7f56f0159740 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.306+0000 7f56f5a3f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cdec0 con 0x7f56f0159b10 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56eeffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.311 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56eeffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32774/0 (socket says 172.21.15.177:32774) 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33926/0 (socket says 172.21.15.177:33926) 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56eeffd700 1 -- 172.21.15.177:0/4188799831 learned_addr learned my addr 172.21.15.177:0/4188799831 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 -- 172.21.15.177:0/4188799831 >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 msgr2=0x7f56f00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 --2- 172.21.15.177:0/4188799831 >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 -- 172.21.15.177:0/4188799831 >> v1:172.21.15.16:6789/0 conn(0x7f56f00cda10 legacy=0x7f56f00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 -- 172.21.15.177:0/4188799831 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00ce1a0 con 0x7f56f0159b10 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56eeffd700 1 --2- 172.21.15.177:0/4188799831 >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:39.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56ef7fe700 1 --2- 172.21.15.177:0/4188799831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009e60 tx=0x7f56e8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8139f7c0848b6f3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56edffb700 1 -- 172.21.15.177:0/4188799831 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e800e070 con 0x7f56f0159b10 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56edffb700 1 -- 172.21.15.177:0/4188799831 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e800bb60 con 0x7f56f0159b10 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.307+0000 7f56edffb700 1 -- 172.21.15.177:0/4188799831 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8012430 con 0x7f56f0159b10 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 -- 172.21.15.177:0/4188799831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 msgr2=0x7f56f00cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/4188799831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f56e8009e60 tx=0x7f56e8009780 comp rx=0 tx=0).stop 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 -- 172.21.15.177:0/4188799831 shutdown_connections 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/4188799831 >> v2:172.21.15.177:3300/0 conn(0x7f56f0159740 0x7f56f00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/4188799831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f0159b10 0x7f56f00cef60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 -- 172.21.15.177:0/4188799831 >> 172.21.15.177:0/4188799831 conn(0x7f56f0099a40 msgr2=0x7f56f006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 -- 172.21.15.177:0/4188799831 shutdown_connections 2023-10-06T11:41:39.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.308+0000 7f56f5a3f700 1 -- 172.21.15.177:0/4188799831 wait complete. 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.309+0000 7f56f5a3f700 1 Processor -- start 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.309+0000 7f56f5a3f700 1 -- start start 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56f0159740 0x7f56f015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56f0159b10 0x7f56f0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f016ac80 con 0x7f56f0159b10 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015b8b0 con 0x7f56f0159740 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56f5a3f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015bb00 con 0x7f56f00cda10 2023-10-06T11:41:39.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56ee7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56f0159740 0x7f56f015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33928/0 (socket says 172.21.15.177:33928) 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 -- 172.21.15.177:0/655112753 learned_addr learned my addr 172.21.15.177:0/655112753 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56ef7fe700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56f0159b10 0x7f56f0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 -- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56f0159740 msgr2=0x7f56f015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56f0159740 0x7f56f015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 -- 172.21.15.177:0/655112753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56f0159b10 msgr2=0x7f56f0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56f0159b10 0x7f56f0152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.310+0000 7f56eeffd700 1 -- 172.21.15.177:0/655112753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e00094f0 con 0x7f56f00cda10 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56eeffd700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56e000ec50 tx=0x7f56e0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000c850 con 0x7f56f00cda10 2023-10-06T11:41:39.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000c9b0 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e8009a20 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000cd50 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f015c3a0 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f56e0005c80 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.311+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f56e0013b80 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.312+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f015bde0 con 0x7f56f00cda10 2023-10-06T11:41:39.318 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.315+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f56e0021060 con 0x7f56f00cda10 2023-10-06T11:41:39.400 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.398+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a175cb92-fb78-4cbb-91f5-5df435916564", "id": 4} v 0) v1 -- 0x7f56f004c380 con 0x7f56f00cda10 2023-10-06T11:41:39.408 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.406+0000 7f56d6ffd700 1 -- 172.21.15.177:0/655112753 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a175cb92-fb78-4cbb-91f5-5df435916564", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f56e0017070 con 0x7f56f00cda10 2023-10-06T11:41:39.408 INFO:teuthology.orchestra.run.smithi177.stdout:4 2023-10-06T11:41:39.409 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 msgr2=0x7f56f016a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.409 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56e000ec50 tx=0x7f56e0009250 comp rx=0 tx=0).stop 2023-10-06T11:41:39.409 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 shutdown_connections 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56f0159b10 0x7f56f0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f56f00cda10 0x7f56f016a740 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 --2- 172.21.15.177:0/655112753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f56f0159740 0x7f56f015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 >> 172.21.15.177:0/655112753 conn(0x7f56f0099a40 msgr2=0x7f56f006e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 shutdown_connections 2023-10-06T11:41:39.410 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.407+0000 7f56f5a3f700 1 -- 172.21.15.177:0/655112753 wait complete. 2023-10-06T11:41:39.421 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new faefcc9b-4187-42af-9488-8acc48afe27e 5 2023-10-06T11:41:39.537 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.534+0000 7ffae99b5700 1 Processor -- start 2023-10-06T11:41:39.537 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.534+0000 7ffae99b5700 1 -- start start 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffae40abc40 0x7ffae406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffae406ee80 0x7ffae406c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae406f300 con 0x7ffae406ee80 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae41599e0 con 0x7ffae40abc40 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae99b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae406da40 con 0x7ffae4159610 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae27fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffae40abc40 0x7ffae406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33942/0 (socket says 172.21.15.177:33942) 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae27fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffae40abc40 0x7ffae406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32786/0 (socket says 172.21.15.177:32786) 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae2ffd700 1 -- 172.21.15.177:0/937868315 learned_addr learned my addr 172.21.15.177:0/937868315 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.535+0000 7ffae37fe700 1 --2- 172.21.15.177:0/937868315 >> v2:172.21.15.16:3300/0 conn(0x7ffae406ee80 0x7ffae406c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 -- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 msgr2=0x7ffae406ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 -- 172.21.15.177:0/937868315 >> v2:172.21.15.16:3300/0 conn(0x7ffae406ee80 msgr2=0x7ffae406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 --2- 172.21.15.177:0/937868315 >> v2:172.21.15.16:3300/0 conn(0x7ffae406ee80 0x7ffae406c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 -- 172.21.15.177:0/937868315 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40ac190 con 0x7ffae40abc40 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae2ffd700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae27fc700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae406e940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffae40658b0 tx=0x7ffacc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26cc6e12536bb24c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.541 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae1ffb700 1 -- 172.21.15.177:0/937868315 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc011630 con 0x7ffae40abc40 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae1ffb700 1 -- 172.21.15.177:0/937868315 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc011790 con 0x7ffae40abc40 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae1ffb700 1 -- 172.21.15.177:0/937868315 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc011b30 con 0x7ffae40abc40 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae99b5700 1 -- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 msgr2=0x7ffae406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.536+0000 7ffae99b5700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae406e940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffae40658b0 tx=0x7ffacc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 -- 172.21.15.177:0/937868315 shutdown_connections 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 --2- 172.21.15.177:0/937868315 >> v2:172.21.15.16:3300/0 conn(0x7ffae406ee80 0x7ffae406c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae4159610 0x7ffae406ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 --2- 172.21.15.177:0/937868315 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae406e940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 -- 172.21.15.177:0/937868315 >> 172.21.15.177:0/937868315 conn(0x7ffae4067b90 msgr2=0x7ffae4067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 -- 172.21.15.177:0/937868315 shutdown_connections 2023-10-06T11:41:39.542 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.537+0000 7ffae99b5700 1 -- 172.21.15.177:0/937868315 wait complete. 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.538+0000 7ffae99b5700 1 Processor -- start 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.538+0000 7ffae99b5700 1 -- start start 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 0x7ffae4151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae414e3b0 con 0x7ffae4159610 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae414e8f0 con 0x7ffae40abc40 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b2180 con 0x7ffae406ee80 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32794/0 (socket says 172.21.15.177:32794) 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae2ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33952/0 (socket says 172.21.15.177:33952) 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 -- 172.21.15.177:0/1675498495 learned_addr learned my addr 172.21.15.177:0/1675498495 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae37fe700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 0x7ffae4151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 -- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 msgr2=0x7ffae414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 -- 172.21.15.177:0/1675498495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 msgr2=0x7ffae4151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 0x7ffae4151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 -- 172.21.15.177:0/1675498495 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc009ae0 con 0x7ffae40abc40 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae37fe700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 0x7ffae4151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:39.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae2ffd700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae27fc700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009f50 tx=0x7ffacc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc011b10 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc0253f0 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc025790 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.539+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffacc009420 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.540+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae40b1850 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.540+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7ffacc02e3f0 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.540+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7ffacc025dc0 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.541+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4069430 con 0x7ffae40abc40 2023-10-06T11:41:39.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.543+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7ffacc033060 con 0x7ffae40abc40 2023-10-06T11:41:39.601 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:39.599+0000 7f62e61e1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:41:39.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.642+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "faefcc9b-4187-42af-9488-8acc48afe27e", "id": 5} v 0) v1 -- 0x7ffae404bb40 con 0x7ffae40abc40 2023-10-06T11:41:39.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.649+0000 7ffadaffd700 1 -- 172.21.15.177:0/1675498495 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "faefcc9b-4187-42af-9488-8acc48afe27e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffacc015020 con 0x7ffae40abc40 2023-10-06T11:41:39.652 INFO:teuthology.orchestra.run.smithi177.stdout:5 2023-10-06T11:41:39.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 msgr2=0x7ffae4150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009f50 tx=0x7ffacc017000 comp rx=0 tx=0).stop 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 shutdown_connections 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae4159610 0x7ffae4151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffae406ee80 0x7ffae414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 --2- 172.21.15.177:0/1675498495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffae40abc40 0x7ffae4150be0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 >> 172.21.15.177:0/1675498495 conn(0x7ffae4067b90 msgr2=0x7ffae40cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 shutdown_connections 2023-10-06T11:41:39.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.650+0000 7ffae99b5700 1 -- 172.21.15.177:0/1675498495 wait complete. 2023-10-06T11:41:39.660 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:39.658+0000 7fca3d281180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T11:41:39.664 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 7e8c078d-e95d-4679-803b-6b59d7076a78 6 2023-10-06T11:41:39.738 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:39.736+0000 7f62e61e1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:41:39.779 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.777+0000 7fd96d7c6700 1 Processor -- start 2023-10-06T11:41:39.779 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.777+0000 7fd96d7c6700 1 -- start start 2023-10-06T11:41:39.781 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96d7c6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9680cef40 0x7fd9680ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96d7c6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9680c64c0 0x7fd9680cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96d7c6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9680cffc0 con 0x7fd9681594a0 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96d7c6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9680cf3b0 con 0x7fd9680c64c0 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96d7c6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9680cf690 con 0x7fd9680cef40 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9680cef40 0x7fd9680ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9680cef40 0x7fd9680ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32810/0 (socket says 172.21.15.177:32810) 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd967fff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd9681594a0 0x7fd9680ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:56422/0 (socket says 172.21.15.177:56422) 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 -- 172.21.15.177:0/3547972646 learned_addr learned my addr 172.21.15.177:0/3547972646 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd96cfc5700 1 --2- 172.21.15.177:0/3547972646 >> v2:172.21.15.16:3300/0 conn(0x7fd9680c64c0 0x7fd9680cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.782 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 -- 172.21.15.177:0/3547972646 >> v2:172.21.15.16:3300/0 conn(0x7fd9680c64c0 msgr2=0x7fd9680cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 --2- 172.21.15.177:0/3547972646 >> v2:172.21.15.16:3300/0 conn(0x7fd9680c64c0 0x7fd9680cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 -- 172.21.15.177:0/3547972646 >> v1:172.21.15.177:6789/0 conn(0x7fd9681594a0 legacy=0x7fd9680ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 -- 172.21.15.177:0/3547972646 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680cf970 con 0x7fd9680cef40 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd966ffd700 1 -- 172.21.15.177:0/3547972646 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453396365 0 0) 0x7fd9680cffc0 con 0x7fd9681594a0 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd9677fe700 1 --2- 172.21.15.177:0/3547972646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd9680ccf60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd968089180 tx=0x7fd958009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41802eee0712a495 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.778+0000 7fd966ffd700 1 -- 172.21.15.177:0/3547972646 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800b860 con 0x7fd9680cef40 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd966ffd700 1 -- 172.21.15.177:0/3547972646 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95800b9c0 con 0x7fd9680cef40 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd966ffd700 1 -- 172.21.15.177:0/3547972646 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95800bd60 con 0x7fd9680cef40 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3547972646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 msgr2=0x7fd9680ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3547972646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd9680ccf60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd968089180 tx=0x7fd958009250 comp rx=0 tx=0).stop 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3547972646 shutdown_connections 2023-10-06T11:41:39.783 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3547972646 >> v2:172.21.15.16:3300/0 conn(0x7fd9680c64c0 0x7fd9680cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.784 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3547972646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd9680ccf60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.784 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3547972646 >> 172.21.15.177:0/3547972646 conn(0x7fd968089f90 msgr2=0x7fd9680998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.784 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3547972646 shutdown_connections 2023-10-06T11:41:39.784 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.779+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3547972646 wait complete. 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.780+0000 7fd96d7c6700 1 Processor -- start 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.780+0000 7fd96d7c6700 1 -- start start 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd96814f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 0x7fd96814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96814fdb0 con 0x7fd9681594a0 2023-10-06T11:41:39.786 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96814d0b0 con 0x7fd9680cef40 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96d7c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96815c310 con 0x7fd9680c64c0 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd967fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd9677fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd96814f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd967fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:33958/0 (socket says 172.21.15.177:33958) 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd967fff700 1 -- 172.21.15.177:0/3060298450 learned_addr learned my addr 172.21.15.177:0/3060298450 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.781+0000 7fd96cfc5700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 0x7fd96814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 -- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 msgr2=0x7fd96814f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd96814f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 -- 172.21.15.177:0/3060298450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 msgr2=0x7fd96814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 0x7fd96814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 -- 172.21.15.177:0/3060298450 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580094f0 con 0x7fd9680c64c0 2023-10-06T11:41:39.787 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd9677fe700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd96814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd96cfc5700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 0x7fd96814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd967fff700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c5e0 tx=0x7fd95c007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0072e0 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c007460 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96815c5f0 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0057f0 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9681504f0 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd95c01c440 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.782+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fd95c005e40 con 0x7fd9680c64c0 2023-10-06T11:41:39.788 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.783+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96814ce80 con 0x7fd9680c64c0 2023-10-06T11:41:39.789 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.785+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd96814ce80 con 0x7fd9680c64c0 2023-10-06T11:41:39.800 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:39.798+0000 7fca3d281180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T11:41:39.816 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:39.815+0000 7f62e61e1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:41:39.870 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.868+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7e8c078d-e95d-4679-803b-6b59d7076a78", "id": 6} v 0) v1 -- 0x7fd96815ca90 con 0x7fd9680c64c0 2023-10-06T11:41:39.875 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.873+0000 7fd94ffff700 1 -- 172.21.15.177:0/3060298450 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e8c078d-e95d-4679-803b-6b59d7076a78", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd95c017070 con 0x7fd9680c64c0 2023-10-06T11:41:39.876 INFO:teuthology.orchestra.run.smithi177.stdout:6 2023-10-06T11:41:39.877 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 msgr2=0x7fd96813d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:39.877 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00c5e0 tx=0x7fd95c007f40 comp rx=0 tx=0).stop 2023-10-06T11:41:39.877 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 shutdown_connections 2023-10-06T11:41:39.877 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9681594a0 0x7fd96814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.877 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9680c64c0 0x7fd96813d040 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.878 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 --2- 172.21.15.177:0/3060298450 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9680cef40 0x7fd96814f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:39.878 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.874+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 >> 172.21.15.177:0/3060298450 conn(0x7fd968089f90 msgr2=0x7fd9680b4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:39.878 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.875+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 shutdown_connections 2023-10-06T11:41:39.878 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:39.875+0000 7fd96d7c6700 1 -- 172.21.15.177:0/3060298450 wait complete. 2023-10-06T11:41:39.878 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:39.876+0000 7fca3d281180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T11:41:39.889 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 1b515708-022a-4aa8-9a84-61b0a99250a9 7 2023-10-06T11:41:40.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.004+0000 7fbd3fba5700 1 Processor -- start 2023-10-06T11:41:40.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.004+0000 7fbd3fba5700 1 -- start start 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3fba5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbd3806cb20 0x7fbd38164570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3fba5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3806bee0 con 0x7fbd381570d0 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3fba5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3806a7f0 con 0x7fbd381574a0 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3fba5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3815a130 con 0x7fbd3806cb20 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3e142700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbd3806cb20 0x7fbd38164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3e142700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbd3806cb20 0x7fbd38164570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32822/0 (socket says 172.21.15.177:32822) 2023-10-06T11:41:40.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3d140700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fbd381574a0 0x7fbd3806c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:56432/0 (socket says 172.21.15.177:56432) 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.005+0000 7fbd3e142700 1 -- 172.21.15.177:0/1354594372 learned_addr learned my addr 172.21.15.177:0/1354594372 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3e142700 1 -- 172.21.15.177:0/1354594372 >> v1:172.21.15.177:6789/0 conn(0x7fbd381574a0 legacy=0x7fbd3806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3e142700 1 -- 172.21.15.177:0/1354594372 >> v1:172.21.15.16:6789/0 conn(0x7fbd381570d0 legacy=0x7fbd380b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3e142700 1 -- 172.21.15.177:0/1354594372 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3815a410 con 0x7fbd3806cb20 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3e142700 1 --2- 172.21.15.177:0/1354594372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd38164570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009f20 tx=0x7fbd30009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7486594c6e89b562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3c93f700 1 -- 172.21.15.177:0/1354594372 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000e070 con 0x7fbd3806cb20 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3c93f700 1 -- 172.21.15.177:0/1354594372 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3000ba80 con 0x7fbd3806cb20 2023-10-06T11:41:40.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.006+0000 7fbd3c93f700 1 -- 172.21.15.177:0/1354594372 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd300092e0 con 0x7fbd3806cb20 2023-10-06T11:41:40.010 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:40.006+0000 7f62e61e1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 -- 172.21.15.177:0/1354594372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 msgr2=0x7fbd38164570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/1354594372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd38164570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009f20 tx=0x7fbd30009840 comp rx=0 tx=0).stop 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 -- 172.21.15.177:0/1354594372 shutdown_connections 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/1354594372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd38164570 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 -- 172.21.15.177:0/1354594372 >> 172.21.15.177:0/1354594372 conn(0x7fbd38068030 msgr2=0x7fbd3806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:40.013 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 -- 172.21.15.177:0/1354594372 shutdown_connections 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.007+0000 7fbd3fba5700 1 -- 172.21.15.177:0/1354594372 wait complete. 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 Processor -- start 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 -- start start 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 0x7fbd38141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd381574a0 0x7fbd380a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381423f0 con 0x7fbd381574a0 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3815b7d0 con 0x7fbd3806cb20 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3fba5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd380acf50 con 0x7fbd381570d0 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3d941700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.008+0000 7fbd3d140700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 0x7fbd38141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.014 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d941700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:32836/0 (socket says 172.21.15.177:32836) 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d941700 1 -- 172.21.15.177:0/2931329705 learned_addr learned my addr 172.21.15.177:0/2931329705 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3e142700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd381574a0 0x7fbd380a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 -- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 msgr2=0x7fbd380af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 -- 172.21.15.177:0/2931329705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd381574a0 msgr2=0x7fbd380a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd381574a0 0x7fbd380a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 -- 172.21.15.177:0/2931329705 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30009ae0 con 0x7fbd381570d0 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d941700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3d140700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 0x7fbd38141eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c590 tx=0x7fbd34007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34005a70 con 0x7fbd381570d0 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd34005bf0 con 0x7fbd381570d0 2023-10-06T11:41:40.015 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd380ad230 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340070d0 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.009+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3814f610 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.010+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fbd3401c440 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.010+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fbd34007700 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.010+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd380afdb0 con 0x7fbd381570d0 2023-10-06T11:41:40.016 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.012+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fbd34017070 con 0x7fbd381570d0 2023-10-06T11:41:40.073 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:40.070+0000 7fca3d281180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:41:40.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.093+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1b515708-022a-4aa8-9a84-61b0a99250a9", "id": 7} v 0) v1 -- 0x7fbd3804c380 con 0x7fbd381570d0 2023-10-06T11:41:40.101 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.099+0000 7fbd2dffb700 1 -- 172.21.15.177:0/2931329705 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b515708-022a-4aa8-9a84-61b0a99250a9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbd3401c880 con 0x7fbd381570d0 2023-10-06T11:41:40.101 INFO:teuthology.orchestra.run.smithi177.stdout:7 2023-10-06T11:41:40.102 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 msgr2=0x7fbd38141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.102 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 0x7fbd38141eb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c590 tx=0x7fbd34007eb0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.102 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 shutdown_connections 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd381574a0 0x7fbd380a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd3806cb20 0x7fbd380af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 --2- 172.21.15.177:0/2931329705 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbd381570d0 0x7fbd38141eb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 >> 172.21.15.177:0/2931329705 conn(0x7fbd38068030 msgr2=0x7fbd381593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 shutdown_connections 2023-10-06T11:41:40.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T11:41:40.100+0000 7fbd3fba5700 1 -- 172.21.15.177:0/2931329705 wait complete. 2023-10-06T11:41:40.114 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T11:41:40.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T11:41:40.116 INFO:tasks.ceph.osd.0:Started 2023-10-06T11:41:40.116 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T11:41:40.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T11:41:40.119 INFO:tasks.ceph.osd.1:Started 2023-10-06T11:41:40.119 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T11:41:40.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T11:41:40.121 INFO:tasks.ceph.osd.2:Started 2023-10-06T11:41:40.121 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T11:41:40.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T11:41:40.126 INFO:tasks.ceph.osd.3:Started 2023-10-06T11:41:40.127 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T11:41:40.127 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T11:41:40.129 INFO:tasks.ceph.osd.4:Started 2023-10-06T11:41:40.129 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T11:41:40.130 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T11:41:40.133 INFO:tasks.ceph.osd.5:Started 2023-10-06T11:41:40.133 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T11:41:40.133 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T11:41:40.135 INFO:tasks.ceph.osd.6:Started 2023-10-06T11:41:40.135 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T11:41:40.135 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T11:41:40.137 INFO:tasks.ceph.osd.7:Started 2023-10-06T11:41:40.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:40.234 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:40.232+0000 7f62e61e1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:41:40.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.273+0000 7fd68883c700 1 Processor -- start 2023-10-06T11:41:40.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.273+0000 7fd68883c700 1 -- start start 2023-10-06T11:41:40.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68883c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd68415cc20 0x7fd6840cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68883c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6840cd6a0 0x7fd6840ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68883c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6840cdb10 con 0x7fd6840cf280 2023-10-06T11:41:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68883c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840b3110 con 0x7fd68415cc20 2023-10-06T11:41:40.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68883c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6840ccb20 con 0x7fd6840cd6a0 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd68415cc20 0x7fd6840cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd68415cc20 0x7fd6840cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38386/0 (socket says 172.21.15.16:38386) 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 -- 172.21.15.16:0/3904906781 learned_addr learned my addr 172.21.15.16:0/3904906781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd681d9b700 1 --2- 172.21.15.16:0/3904906781 >> v2:172.21.15.177:3300/0 conn(0x7fd6840cd6a0 0x7fd6840ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 -- 172.21.15.16:0/3904906781 >> v2:172.21.15.177:3300/0 conn(0x7fd6840cd6a0 msgr2=0x7fd6840ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:40.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 --2- 172.21.15.16:0/3904906781 >> v2:172.21.15.177:3300/0 conn(0x7fd6840cd6a0 0x7fd6840ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 -- 172.21.15.16:0/3904906781 >> v1:172.21.15.177:6789/0 conn(0x7fd6840cf280 legacy=0x7fd68415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 -- 172.21.15.16:0/3904906781 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6840cbf10 con 0x7fd68415cc20 2023-10-06T11:41:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.274+0000 7fd68259c700 1 --2- 172.21.15.16:0/3904906781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd68415cc20 0x7fd6840cd160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6840cb150 tx=0x7fd674009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8742b6a33cc08a67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68159a700 1 -- 172.21.15.16:0/3904906781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd674009e70 con 0x7fd68415cc20 2023-10-06T11:41:40.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68159a700 1 -- 172.21.15.16:0/3904906781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67400b860 con 0x7fd68415cc20 2023-10-06T11:41:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68159a700 1 -- 172.21.15.16:0/3904906781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400bc20 con 0x7fd68415cc20 2023-10-06T11:41:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 -- 172.21.15.16:0/3904906781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd68415cc20 msgr2=0x7fd6840cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 --2- 172.21.15.16:0/3904906781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd68415cc20 0x7fd6840cd160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd6840cb150 tx=0x7fd674009250 comp rx=0 tx=0).stop 2023-10-06T11:41:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 -- 172.21.15.16:0/3904906781 shutdown_connections 2023-10-06T11:41:40.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 --2- 172.21.15.16:0/3904906781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd68415cc20 0x7fd6840cd160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 --2- 172.21.15.16:0/3904906781 >> v2:172.21.15.177:3300/0 conn(0x7fd6840cd6a0 0x7fd6840ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 -- 172.21.15.16:0/3904906781 >> 172.21.15.16:0/3904906781 conn(0x7fd684099b40 msgr2=0x7fd68406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 -- 172.21.15.16:0/3904906781 shutdown_connections 2023-10-06T11:41:40.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.275+0000 7fd68883c700 1 -- 172.21.15.16:0/3904906781 wait complete. 2023-10-06T11:41:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.276+0000 7fd68883c700 1 Processor -- start 2023-10-06T11:41:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.276+0000 7fd68883c700 1 -- start start 2023-10-06T11:41:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6840cf280 0x7fd6840b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd68415cc20 0x7fd6840b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:40.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6840b5090 con 0x7fd6840cd6a0 2023-10-06T11:41:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd684153ca0 con 0x7fd6840cf280 2023-10-06T11:41:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68883c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd684155670 con 0x7fd68415cc20 2023-10-06T11:41:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38396/0 (socket says 172.21.15.16:38396) 2023-10-06T11:41:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd68259c700 1 -- 172.21.15.16:0/808857807 learned_addr learned my addr 172.21.15.16:0/808857807 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd681d9b700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6840cf280 0x7fd6840b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.277+0000 7fd682d9d700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd68415cc20 0x7fd6840b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 -- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd68415cc20 msgr2=0x7fd6840b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd68415cc20 0x7fd6840b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 -- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6840cf280 msgr2=0x7fd6840b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6840cf280 0x7fd6840b56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 -- 172.21.15.16:0/808857807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740094f0 con 0x7fd6840cd6a0 2023-10-06T11:41:40.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68259c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd684089390 tx=0x7fd674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:40.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67401f070 con 0x7fd6840cd6a0 2023-10-06T11:41:40.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd674002f30 con 0x7fd6840cd6a0 2023-10-06T11:41:40.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6840b4510 con 0x7fd6840cd6a0 2023-10-06T11:41:40.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67400b9d0 con 0x7fd6840cd6a0 2023-10-06T11:41:40.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6840b4ad0 con 0x7fd6840cd6a0 2023-10-06T11:41:40.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd674023070 con 0x7fd6840cd6a0 2023-10-06T11:41:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.278+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fd674025b00 con 0x7fd6840cd6a0 2023-10-06T11:41:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.279+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68404c380 con 0x7fd6840cd6a0 2023-10-06T11:41:40.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.281+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd67402a070 con 0x7fd6840cd6a0 2023-10-06T11:41:40.292 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:40.283+0000 7fca3d281180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T11:41:40.316 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:40.314+0000 7f62e61e1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:41:40.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.352+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6840cc1f0 con 0x7fd6840cd6a0 2023-10-06T11:41:40.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.353+0000 7fd6727fc700 1 -- 172.21.15.16:0/808857807 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fd674015020 con 0x7fd6840cd6a0 2023-10-06T11:41:40.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:40.355 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:40.096177+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:40.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 msgr2=0x7fd684153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:40.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd684089390 tx=0x7fd674017000 comp rx=0 tx=0).stop 2023-10-06T11:41:40.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 shutdown_connections 2023-10-06T11:41:40.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6840cd6a0 0x7fd684153760 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd68415cc20 0x7fd6840b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 --2- 172.21.15.16:0/808857807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6840cf280 0x7fd6840b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:40.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 >> 172.21.15.16:0/808857807 conn(0x7fd684099b40 msgr2=0x7fd6841544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:40.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 shutdown_connections 2023-10-06T11:41:40.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:40.354+0000 7fd68883c700 1 -- 172.21.15.16:0/808857807 wait complete. 2023-10-06T11:41:40.361 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:40.359+0000 7fca3d281180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T11:41:40.369 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T11:41:40.369 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T11:41:40.456 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:40.454+0000 7f62e61e1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:41:40.497 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:40.494+0000 7fca3d281180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T11:41:40.556 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:40.554+0000 7f62e61e1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:41:40.591 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:40.589+0000 7fca3d281180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T11:41:40.727 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-10-06T11:41:40.725+0000 7f1d64a3c540 -1 Falling back to public interface 2023-10-06T11:41:40.728 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T11:41:40.725+0000 7fad48775540 -1 Falling back to public interface 2023-10-06T11:41:40.731 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T11:41:40.730+0000 7ff573d8d540 -1 Falling back to public interface 2023-10-06T11:41:40.735 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T11:41:40.733+0000 7f17e52f8540 -1 Falling back to public interface 2023-10-06T11:41:40.740 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-10-06T11:41:40.738+0000 7fa20846d540 -1 Falling back to public interface 2023-10-06T11:41:40.748 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-10-06T11:41:40.746+0000 7f1a12472540 -1 Falling back to public interface 2023-10-06T11:41:40.748 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T11:41:40.746+0000 7fc17f40c540 -1 Falling back to public interface 2023-10-06T11:41:40.751 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-10-06T11:41:40.747+0000 7f10c93ac540 -1 Falling back to public interface 2023-10-06T11:41:41.092 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:41.090+0000 7f62e61e1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:41:41.116 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:41.114+0000 7fca3d281180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T11:41:41.172 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:41.170+0000 7f62e61e1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:41:41.175 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:41.193 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:41.189+0000 7fca3d281180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T11:41:41.255 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:41.253+0000 7f62e61e1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:41:41.267 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:41.265+0000 7fca3d281180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T11:41:41.286 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.284+0000 7f918a890700 1 Processor -- start 2023-10-06T11:41:41.286 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.284+0000 7f918a890700 1 -- start start 2023-10-06T11:41:41.288 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f918a890700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184068e20 0x7f918406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:41.288 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f918a890700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91840b0440 0x7f91840b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:41.288 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f918a890700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918406eba0 con 0x7f9184067600 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f918a890700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918406f7b0 con 0x7f91840b0440 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f918a890700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918406ee80 con 0x7f9184068e20 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91840b0440 0x7f91840b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9183fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9184067600 0x7f91840679d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53644/0 (socket says 172.21.15.16:53644) 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9183fff700 1 -- 172.21.15.16:0/3595286767 learned_addr learned my addr 172.21.15.16:0/3595286767 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f91837fe700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184068e20 0x7f918406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 -- 172.21.15.16:0/3595286767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184068e20 msgr2=0x7f918406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184068e20 0x7f918406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 -- 172.21.15.16:0/3595286767 >> v1:172.21.15.16:6789/0 conn(0x7f9184067600 legacy=0x7f91840679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 -- 172.21.15.16:0/3595286767 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918406f160 con 0x7f91840b0440 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9182ffd700 1 -- 172.21.15.16:0/3595286767 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149364990 0 0) 0x7f918406eba0 con 0x7f9184067600 2023-10-06T11:41:41.289 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9188e2d700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840b0440 0x7f91840b08c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e60 tx=0x7f917c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b0cff45f882c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.285+0000 7f9182ffd700 1 -- 172.21.15.16:0/3595286767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00e070 con 0x7f91840b0440 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f9182ffd700 1 -- 172.21.15.16:0/3595286767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c00bb60 con 0x7f91840b0440 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f9182ffd700 1 -- 172.21.15.16:0/3595286767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c012430 con 0x7f91840b0440 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 -- 172.21.15.16:0/3595286767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840b0440 msgr2=0x7f91840b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840b0440 0x7f91840b08c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f917c009e60 tx=0x7f917c009780 comp rx=0 tx=0).stop 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 -- 172.21.15.16:0/3595286767 shutdown_connections 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91840b0440 0x7f91840b08c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 --2- 172.21.15.16:0/3595286767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184068e20 0x7f918406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 -- 172.21.15.16:0/3595286767 >> 172.21.15.16:0/3595286767 conn(0x7f918406bff0 msgr2=0x7f9184068710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 -- 172.21.15.16:0/3595286767 shutdown_connections 2023-10-06T11:41:41.290 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.286+0000 7f918a890700 1 -- 172.21.15.16:0/3595286767 wait complete. 2023-10-06T11:41:41.292 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.287+0000 7f918a890700 1 Processor -- start 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.287+0000 7f918a890700 1 -- start start 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184067600 0x7f91840c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91840b0440 0x7f9184158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c5e40 con 0x7f9184068e20 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c6b20 con 0x7f91840b0440 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f918a890700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c55a0 con 0x7f9184067600 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f91837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f91837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38428/0 (socket says 172.21.15.16:38428) 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f91837fe700 1 -- 172.21.15.16:0/3497165365 learned_addr learned my addr 172.21.15.16:0/3497165365 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:41.293 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f9183fff700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184067600 0x7f91840c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.288+0000 7f9188e2d700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91840b0440 0x7f9184158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 -- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184067600 msgr2=0x7f91840c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184067600 0x7f91840c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 -- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91840b0440 msgr2=0x7f9184158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91840b0440 0x7f9184158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 -- 172.21.15.16:0/3497165365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009a20 con 0x7f9184068e20 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f91837fe700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f917400ec80 tx=0x7f91740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400c850 con 0x7f9184068e20 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91840c5880 con 0x7f9184068e20 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91840c52b0 con 0x7f9184068e20 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400cbc0 con 0x7f9184068e20 2023-10-06T11:41:41.294 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.289+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400cd30 con 0x7f9184068e20 2023-10-06T11:41:41.295 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.290+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f9174005c80 con 0x7f9184068e20 2023-10-06T11:41:41.295 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.290+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f9174006700 con 0x7f9184068e20 2023-10-06T11:41:41.295 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.290+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184063f60 con 0x7f9184068e20 2023-10-06T11:41:41.295 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.292+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f917403b6d0 con 0x7f9184068e20 2023-10-06T11:41:41.365 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.363+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91840afba0 con 0x7f9184068e20 2023-10-06T11:41:41.366 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.364+0000 7f916bfff700 1 -- 172.21.15.16:0/3497165365 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f917402a070 con 0x7f9184068e20 2023-10-06T11:41:41.366 INFO:teuthology.misc.health.smithi016.stdout: 2023-10-06T11:41:41.366 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:40.096177+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:41.367 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 msgr2=0x7f9184144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:41.367 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f917400ec80 tx=0x7f91740091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.367 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 shutdown_connections 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91840b0440 0x7f9184158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9184067600 0x7f91840c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 --2- 172.21.15.16:0/3497165365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9184068e20 0x7f9184144c20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 >> 172.21.15.16:0/3497165365 conn(0x7f918406bff0 msgr2=0x7f9184156050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 shutdown_connections 2023-10-06T11:41:41.368 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:41.365+0000 7f918a890700 1 -- 172.21.15.16:0/3497165365 wait complete. 2023-10-06T11:41:41.378 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T11:41:41.845 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:41.843+0000 7fca3d281180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T11:41:41.877 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:41.875+0000 7f62e61e1180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T11:41:41.982 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:41.980+0000 7fca3d281180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:41:42.020 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:42.018+0000 7f62e61e1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T11:41:42.072 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:42.070+0000 7fca3d281180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:41:42.116 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:42.114+0000 7f62e61e1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T11:41:42.320 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:42.317+0000 7fca3d281180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T11:41:42.373 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:42.371+0000 7f62e61e1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T11:41:42.398 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:42.396+0000 7fca3d281180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:41:42.454 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:42.452+0000 7f62e61e1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T11:41:43.227 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.225+0000 7fca3d281180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T11:41:43.300 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.298+0000 7fca3d281180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:41:43.324 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.322+0000 7f62e61e1180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T11:41:43.373 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.371+0000 7fca3d281180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:41:43.400 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.398+0000 7f62e61e1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T11:41:43.482 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.480+0000 7f62e61e1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T11:41:43.545 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.542+0000 7fca3d281180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:41:43.656 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.654+0000 7fca3d281180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:41:43.663 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.662+0000 7f62e61e1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T11:41:43.751 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.749+0000 7f62e61e1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T11:41:43.807 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.805+0000 7fca3d281180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:41:43.882 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.880+0000 7fca3d281180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:41:43.909 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.906+0000 7f62e61e1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T11:41:43.961 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:43.959+0000 7fca3d281180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:41:43.997 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:43.995+0000 7f62e61e1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T11:41:44.039 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:44.037+0000 7fca3d281180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:41:44.075 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:44.073+0000 7f62e61e1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T11:41:44.154 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-10-06T11:41:44.152+0000 7f62e61e1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T11:41:44.237 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-10-06T11:41:44.235+0000 7f1d64a3c540 -1 osd.1 0 log_to_monitors true 2023-10-06T11:41:44.245 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-10-06T11:41:44.243+0000 7f1a12472540 -1 osd.3 0 log_to_monitors true 2023-10-06T11:41:44.250 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-10-06T11:41:44.248+0000 7fa20846d540 -1 osd.0 0 log_to_monitors true 2023-10-06T11:41:44.252 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T11:41:44.250+0000 7f17e52f8540 -1 osd.7 0 log_to_monitors true 2023-10-06T11:41:44.253 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T11:41:44.251+0000 7ff573d8d540 -1 osd.4 0 log_to_monitors true 2023-10-06T11:41:44.254 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T11:41:44.252+0000 7fad48775540 -1 osd.5 0 log_to_monitors true 2023-10-06T11:41:44.263 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T11:41:44.261+0000 7fc17f40c540 -1 osd.6 0 log_to_monitors true 2023-10-06T11:41:44.265 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-10-06T11:41:44.263+0000 7f10c93ac540 -1 osd.2 0 log_to_monitors true 2023-10-06T11:41:45.067 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:45.065+0000 7fc9f6681700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T11:41:45.067 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:41:45.065+0000 7fc9f6681700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T11:41:47.468 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-10-06T11:41:47.466+0000 7f10bffce700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T11:41:47.473 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-10-06T11:41:47.471+0000 7f10b8dc5700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:47.490 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-10-06T11:41:47.488+0000 7f1d5b65e700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T11:41:47.495 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-10-06T11:41:47.493+0000 7f1d54455700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:47.539 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-10-06T11:41:47.537+0000 7fa1ff08f700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T11:41:47.543 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-10-06T11:41:47.542+0000 7fa1f7e86700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:47.546 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-10-06T11:41:47.544+0000 7f1a09094700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T11:41:47.550 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-10-06T11:41:47.548+0000 7f1a01e8b700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:48.133 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T11:41:48.131+0000 7f17dbf1a700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T11:41:48.137 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T11:41:48.135+0000 7f17d4d11700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:48.145 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T11:41:48.143+0000 7fad40b9a700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T11:41:48.150 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T11:41:48.148+0000 7fad3818e700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:48.150 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T11:41:48.149+0000 7fc17602e700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T11:41:48.154 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T11:41:48.152+0000 7fc16ee25700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:48.161 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T11:41:48.159+0000 7ff56a9af700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T11:41:48.164 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T11:41:48.162+0000 7ff5637a6700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T11:41:48.184 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:48.303 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.301+0000 7f2e7ae78700 1 Processor -- start 2023-10-06T11:41:48.303 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.301+0000 7f2e7ae78700 1 -- start start 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e7ae78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e74159320 0x7f2e74067600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e7ae78700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e741596f0 0x7f2e740ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e7ae78700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e740689d0 con 0x7f2e740aeed0 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e7ae78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e74067b40 con 0x7f2e74159320 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e7ae78700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e74067dc0 con 0x7f2e741596f0 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e79e76700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e74159320 0x7f2e74067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e79e76700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2e74159320 0x7f2e74067600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57164/0 (socket says 172.21.15.16:57164) 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e79e76700 1 -- 172.21.15.16:0/3731666901 learned_addr learned my addr 172.21.15.16:0/3731666901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.302+0000 7f2e79675700 1 --2- 172.21.15.16:0/3731666901 >> v2:172.21.15.177:3300/0 conn(0x7f2e741596f0 0x7f2e740ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.306 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e79e76700 1 -- 172.21.15.16:0/3731666901 >> v2:172.21.15.177:3300/0 conn(0x7f2e741596f0 msgr2=0x7f2e740ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e79e76700 1 --2- 172.21.15.16:0/3731666901 >> v2:172.21.15.177:3300/0 conn(0x7f2e741596f0 0x7f2e740ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e79e76700 1 -- 172.21.15.16:0/3731666901 >> v1:172.21.15.177:6789/0 conn(0x7f2e740aeed0 legacy=0x7f2e7415d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e79e76700 1 -- 172.21.15.16:0/3731666901 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740680a0 con 0x7f2e74159320 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e79e76700 1 --2- 172.21.15.16:0/3731666901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74067600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e7406d9a0 tx=0x7f2e64009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d3d22d3fd5842 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e78e74700 1 -- 172.21.15.16:0/3731666901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6400e070 con 0x7f2e74159320 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e78e74700 1 -- 172.21.15.16:0/3731666901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6400ba10 con 0x7f2e74159320 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e78e74700 1 -- 172.21.15.16:0/3731666901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e640092e0 con 0x7f2e74159320 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e7ae78700 1 -- 172.21.15.16:0/3731666901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 msgr2=0x7f2e74067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/3731666901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74067600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e7406d9a0 tx=0x7f2e64009840 comp rx=0 tx=0).stop 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e7ae78700 1 -- 172.21.15.16:0/3731666901 shutdown_connections 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.303+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/3731666901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74067600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.307 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.304+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/3731666901 >> v2:172.21.15.177:3300/0 conn(0x7f2e741596f0 0x7f2e740ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.308 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.304+0000 7f2e7ae78700 1 -- 172.21.15.16:0/3731666901 >> 172.21.15.16:0/3731666901 conn(0x7f2e740cd350 msgr2=0x7f2e740c9350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:48.308 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.304+0000 7f2e7ae78700 1 -- 172.21.15.16:0/3731666901 shutdown_connections 2023-10-06T11:41:48.308 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.304+0000 7f2e7ae78700 1 -- 172.21.15.16:0/3731666901 wait complete. 2023-10-06T11:41:48.310 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 Processor -- start 2023-10-06T11:41:48.310 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 -- start start 2023-10-06T11:41:48.310 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e740aeed0 0x7f2e7413f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e741596f0 0x7f2e74157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7414dd70 con 0x7f2e74159320 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c5be0 con 0x7f2e741596f0 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.305+0000 7f2e7ae78700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e740c5ec0 con 0x7f2e740aeed0 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57176/0 (socket says 172.21.15.16:57176) 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 -- 172.21.15.16:0/1691852460 learned_addr learned my addr 172.21.15.16:0/1691852460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e7a677700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e741596f0 0x7f2e74157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79e76700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e740aeed0 0x7f2e7413f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 -- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e740aeed0 msgr2=0x7f2e7413f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.311 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e740aeed0 0x7f2e7413f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 -- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e741596f0 msgr2=0x7f2e74157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e741596f0 0x7f2e74157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 -- 172.21.15.16:0/1691852460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64009ae0 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e79675700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e68006b00 tx=0x7f2e68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e680125f0 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e68012770 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68012b30 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e740c61a0 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.306+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7413f7e0 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.307+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7414e050 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.307+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2e6801e460 con 0x7f2e74159320 2023-10-06T11:41:48.312 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.308+0000 7f2e71ffb700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 0x7f2e60069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.313 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.308+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f2e68083c70 con 0x7f2e74159320 2023-10-06T11:41:48.313 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.308+0000 7f2e79e76700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 0x7f2e60069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.313 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.308+0000 7f2e79e76700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 0x7f2e60069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009880 tx=0x7f2e64017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.314 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.312+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e68051d50 con 0x7f2e74159320 2023-10-06T11:41:48.516 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.514+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e7404c380 con 0x7f2e74159320 2023-10-06T11:41:48.516 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.515+0000 7f2e71ffb700 1 -- 172.21.15.16:0/1691852460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f2e68055370 con 0x7f2e74159320 2023-10-06T11:41:48.517 INFO:teuthology.misc.health.smithi016.stdout: 2023-10-06T11:41:48.517 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:48.441522+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:48.519 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 msgr2=0x7f2e60069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.519 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 0x7f2e60069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009880 tx=0x7f2e64017000 comp rx=0 tx=0).stop 2023-10-06T11:41:48.519 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 msgr2=0x7f2e74156c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.519 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2e68006b00 tx=0x7f2e68008040 comp rx=0 tx=0).stop 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 shutdown_connections 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2e60066ea0 0x7f2e60069350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e741596f0 0x7f2e74157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e740aeed0 0x7f2e7413f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 --2- 172.21.15.16:0/1691852460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e74159320 0x7f2e74156c50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 >> 172.21.15.16:0/1691852460 conn(0x7f2e740cd350 msgr2=0x7f2e7406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 shutdown_connections 2023-10-06T11:41:48.520 INFO:teuthology.misc.health.smithi016.stderr:2023-10-06T11:41:48.517+0000 7f2e7ae78700 1 -- 172.21.15.16:0/1691852460 wait complete. 2023-10-06T11:41:48.530 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T11:41:48.530 INFO:tasks.ceph:Creating RBD pool 2023-10-06T11:41:48.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T11:41:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.647+0000 7f455c2a1700 1 Processor -- start 2023-10-06T11:41:48.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 -- start start 2023-10-06T11:41:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f45540cda10 0x7f45540b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45540cead0 con 0x7f4554159740 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45540affc0 con 0x7f45540cda10 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.648+0000 7f455c2a1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45540cdec0 con 0x7f4554159b10 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455a83e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455a83e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41834/0 (socket says 172.21.15.16:41834) 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455a83e700 1 -- 172.21.15.16:0/3822098271 learned_addr learned my addr 172.21.15.16:0/3822098271 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 --2- 172.21.15.16:0/3822098271 >> v2:172.21.15.177:3300/0 conn(0x7f45540cda10 0x7f45540b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 -- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 msgr2=0x7f45540cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 -- 172.21.15.16:0/3822098271 >> v1:172.21.15.177:6789/0 conn(0x7f4554159740 legacy=0x7f45540b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455903b700 1 -- 172.21.15.16:0/3822098271 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406682174 0 0) 0x7f45540cead0 con 0x7f4554159740 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 -- 172.21.15.16:0/3822098271 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45540ce1a0 con 0x7f45540cda10 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455a83e700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455983c700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b40a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45480098e0 tx=0x7f45480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b60ca2f0970af2f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.649+0000 7f455903b700 1 -- 172.21.15.16:0/3822098271 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800b640 con 0x7f45540cda10 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455903b700 1 -- 172.21.15.16:0/3822098271 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454800b7a0 con 0x7f45540cda10 2023-10-06T11:41:48.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455903b700 1 -- 172.21.15.16:0/3822098271 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800bb40 con 0x7f45540cda10 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 -- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 msgr2=0x7f45540b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b40a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45480098e0 tx=0x7f45480091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 -- 172.21.15.16:0/3822098271 shutdown_connections 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f45540cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 --2- 172.21.15.16:0/3822098271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b40a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 -- 172.21.15.16:0/3822098271 >> 172.21.15.16:0/3822098271 conn(0x7f4554099a40 msgr2=0x7f455406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 -- 172.21.15.16:0/3822098271 shutdown_connections 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.650+0000 7f455c2a1700 1 -- 172.21.15.16:0/3822098271 wait complete. 2023-10-06T11:41:48.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.651+0000 7f455c2a1700 1 Processor -- start 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.651+0000 7f455c2a1700 1 -- start start 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f455414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455409df30 con 0x7f4554159740 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455409e210 con 0x7f45540cda10 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455c2a1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455409a450 con 0x7f4554159b10 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455983c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455983c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57180/0 (socket says 172.21.15.16:57180) 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455983c700 1 -- 172.21.15.16:0/2021984516 learned_addr learned my addr 172.21.15.16:0/2021984516 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455a03d700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.652+0000 7f455a83e700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f455414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 -- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 msgr2=0x7f455414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f455414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 -- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 msgr2=0x7f45540b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 -- 172.21.15.16:0/2021984516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0094f0 con 0x7f4554159740 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455a83e700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f455414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455a03d700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455983c700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f455406d350 tx=0x7f4548017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548000d10 con 0x7f4554159740 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454800bb20 con 0x7f4554159740 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4548009420 con 0x7f4554159740 2023-10-06T11:41:48.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548011670 con 0x7f4554159740 2023-10-06T11:41:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.653+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455409aa90 con 0x7f4554159740 2023-10-06T11:41:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.654+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455409ac20 con 0x7f4554159740 2023-10-06T11:41:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.654+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f4548011a20 con 0x7f4554159740 2023-10-06T11:41:48.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.655+0000 7f45527fc700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 0x7f4540069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.655+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f4548085460 con 0x7f4554159740 2023-10-06T11:41:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.655+0000 7f455a03d700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 0x7f4540069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:48.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.655+0000 7f455a03d700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 0x7f4540069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4544001560 tx=0x7f4544009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:48.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.659+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4548052d40 con 0x7f4554159740 2023-10-06T11:41:48.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:48.851+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f455404c380 con 0x7f4554159740 2023-10-06T11:41:49.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.444+0000 7f45527fc700 1 -- 172.21.15.16:0/2021984516 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f4548056360 con 0x7f4554159740 2023-10-06T11:41:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 msgr2=0x7f4540069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 0x7f4540069350 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4544001560 tx=0x7f4544009040 comp rx=0 tx=0).stop 2023-10-06T11:41:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 msgr2=0x7f455409d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f455406d350 tx=0x7f4548017000 comp rx=0 tx=0).stop 2023-10-06T11:41:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 shutdown_connections 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4540066ea0 0x7f4540069350 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4554159b10 0x7f455414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45540cda10 0x7f45540b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 --2- 172.21.15.16:0/2021984516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4554159740 0x7f455409d4c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 >> 172.21.15.16:0/2021984516 conn(0x7f4554099a40 msgr2=0x7f45541564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 shutdown_connections 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.447+0000 7f455c2a1700 1 -- 172.21.15.16:0/2021984516 wait complete. 2023-10-06T11:41:49.450 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2023-10-06T11:41:49.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T11:41:49.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.584+0000 7f845ce5e700 1 Processor -- start 2023-10-06T11:41:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.584+0000 7f845ce5e700 1 -- start start 2023-10-06T11:41:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.585+0000 7f845ce5e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.585+0000 7f845ce5e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84580cd900 0x7f84580cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.585+0000 7f845ce5e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84580cca70 con 0x7f84580cb3f0 2023-10-06T11:41:49.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.585+0000 7f845ce5e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84580cf950 con 0x7f84580cd900 2023-10-06T11:41:49.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.585+0000 7f845ce5e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580cbe60 con 0x7f84580b2a40 2023-10-06T11:41:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f8456d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84580cd900 0x7f84580cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41844/0 (socket says 172.21.15.16:41844) 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 -- 172.21.15.16:0/1822043001 learned_addr learned my addr 172.21.15.16:0/1822043001 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 -- 172.21.15.16:0/1822043001 >> v2:172.21.15.177:3300/0 conn(0x7f84580cd900 msgr2=0x7f84580cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 --2- 172.21.15.16:0/1822043001 >> v2:172.21.15.177:3300/0 conn(0x7f84580cd900 0x7f84580cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 -- 172.21.15.16:0/1822043001 >> v1:172.21.15.177:6789/0 conn(0x7f84580cb3f0 legacy=0x7f84580cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 -- 172.21.15.16:0/1822043001 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84580cc140 con 0x7f84580b2a40 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845659c700 1 --2- 172.21.15.16:0/1822043001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8450009990 tx=0x7f8450009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4105d9e54a32747d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:49.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845559a700 1 -- 172.21.15.16:0/1822043001 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450009e80 con 0x7f84580b2a40 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845559a700 1 -- 172.21.15.16:0/1822043001 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450005c10 con 0x7f84580b2a40 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.586+0000 7f845559a700 1 -- 172.21.15.16:0/1822043001 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000b650 con 0x7f84580b2a40 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 -- 172.21.15.16:0/1822043001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 msgr2=0x7f84580b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 --2- 172.21.15.16:0/1822043001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8450009990 tx=0x7f8450009220 comp rx=0 tx=0).stop 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 -- 172.21.15.16:0/1822043001 shutdown_connections 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 --2- 172.21.15.16:0/1822043001 >> v2:172.21.15.177:3300/0 conn(0x7f84580cd900 0x7f84580cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 --2- 172.21.15.16:0/1822043001 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580b2a40 0x7f84580b2e10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 -- 172.21.15.16:0/1822043001 >> 172.21.15.16:0/1822043001 conn(0x7f845808a230 msgr2=0x7f8458099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 -- 172.21.15.16:0/1822043001 shutdown_connections 2023-10-06T11:41:49.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.587+0000 7f845ce5e700 1 -- 172.21.15.16:0/1822043001 wait complete. 2023-10-06T11:41:49.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.588+0000 7f845ce5e700 1 Processor -- start 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.588+0000 7f845ce5e700 1 -- start start 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580cb3f0 0x7f84580b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84580cd900 0x7f84580c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b49e0 con 0x7f84580b2a40 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b4cc0 con 0x7f84580cd900 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845ce5e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b3e70 con 0x7f84580cb3f0 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57190/0 (socket says 172.21.15.16:57190) 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 -- 172.21.15.16:0/3247321064 learned_addr learned my addr 172.21.15.16:0/3247321064 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f8456d9d700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84580cd900 0x7f84580c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f8455d9b700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580cb3f0 0x7f84580b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 -- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580cb3f0 msgr2=0x7f84580b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580cb3f0 0x7f84580b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 -- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84580cd900 msgr2=0x7f84580c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84580cd900 0x7f84580c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.589+0000 7f845659c700 1 -- 172.21.15.16:0/3247321064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450009480 con 0x7f84580b2a40 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f845659c700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f845806d3f0 tx=0x7f8450016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450011c30 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450011d90 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845001c660 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84580b4150 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.590+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845814b760 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.591+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8458063e70 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.591+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f845001ca10 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.592+0000 7f844e7fc700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 0x7f8438069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.592+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4809+0+0 (secure 0 0 0) 0x7f8450085a00 con 0x7f84580b2a40 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.592+0000 7f8455d9b700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 0x7f8438069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:49.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.592+0000 7f8455d9b700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 0x7f8438069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8440003a80 tx=0x7f8440007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.596+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8450053030 con 0x7f84580b2a40 2023-10-06T11:41:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:49.787+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f84580cf550 con 0x7f84580b2a40 2023-10-06T11:41:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.451+0000 7f844e7fc700 1 -- 172.21.15.16:0/3247321064 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8450056650 con 0x7f84580b2a40 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.454+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 msgr2=0x7f8438069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.454+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 0x7f8438069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8440003a80 tx=0x7f8440007ec0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.454+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 msgr2=0x7f845814b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.454+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f845806d3f0 tx=0x7f8450016000 comp rx=0 tx=0).stop 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 shutdown_connections 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84580b2a40 0x7f845814b220 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8438066ea0 0x7f8438069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84580cd900 0x7f84580c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 --2- 172.21.15.16:0/3247321064 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84580cb3f0 0x7f84580b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 >> 172.21.15.16:0/3247321064 conn(0x7f845808a230 msgr2=0x7f8458156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 shutdown_connections 2023-10-06T11:41:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.455+0000 7f845ce5e700 1 -- 172.21.15.16:0/3247321064 wait complete. 2023-10-06T11:41:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T11:41:50.471 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T11:41:50.472 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:41:50.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:41:50.474 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:41:50.474 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:41:50.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:41:50.476 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:41:50.476 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:41:50.477 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:41:50.479 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:41:50.479 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:41:50.479 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:41:50.481 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:41:50.481 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T11:41:50.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:50.557 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-10-06T11:41:50.557 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-10-06T11:41:50.559 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T11:41:50.559 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T11:41:50.559 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-10-06T11:41:50.558+0000 7f65a9010700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T11:41:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.610+0000 7f3cf39ba700 1 Processor -- start 2023-10-06T11:41:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.610+0000 7f3cf39ba700 1 -- start start 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf39ba700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cec15cc20 0x7f3cec0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf39ba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec0cdb10 con 0x7f3cec0cf450 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf39ba700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec0ccb20 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf39ba700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cec0b3110 con 0x7f3cec15cc20 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cec15cc20 0x7f3cec0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cec15cc20 0x7f3cec0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57220/0 (socket says 172.21.15.16:57220) 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 -- 172.21.15.16:0/4108383112 learned_addr learned my addr 172.21.15.16:0/4108383112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1f57700 1 --1- 172.21.15.16:0/4108383112 >> v1:172.21.15.16:6789/0 conn(0x7f3cec0cf450 0x7f3cec0c5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46112/0 (socket says 172.21.15.16:46112) 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 -- 172.21.15.16:0/4108383112 >> v1:172.21.15.177:6789/0 conn(0x7f3cec0cd6a0 legacy=0x7f3cec0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 -- 172.21.15.16:0/4108383112 >> v1:172.21.15.16:6789/0 conn(0x7f3cec0cf450 legacy=0x7f3cec0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 -- 172.21.15.16:0/4108383112 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0cf880 con 0x7f3cec15cc20 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3ce3fff700 1 -- 172.21.15.16:0/4108383112 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336146846 0 0) 0x7f3cec0cdb10 con 0x7f3cec0cf450 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.611+0000 7f3cf1756700 1 --2- 172.21.15.16:0/4108383112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec15cc20 0x7f3cec0cd160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80099b0 tx=0x7f3ce8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ae0c3cbc937f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3ce3fff700 1 -- 172.21.15.16:0/4108383112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800b860 con 0x7f3cec15cc20 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3ce3fff700 1 -- 172.21.15.16:0/4108383112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce800b9c0 con 0x7f3cec15cc20 2023-10-06T11:41:50.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3ce3fff700 1 -- 172.21.15.16:0/4108383112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800bd80 con 0x7f3cec15cc20 2023-10-06T11:41:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 -- 172.21.15.16:0/4108383112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec15cc20 msgr2=0x7f3cec0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/4108383112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec15cc20 0x7f3cec0cd160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ce80099b0 tx=0x7f3ce8009250 comp rx=0 tx=0).stop 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 -- 172.21.15.16:0/4108383112 shutdown_connections 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/4108383112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec15cc20 0x7f3cec0cd160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 -- 172.21.15.16:0/4108383112 >> 172.21.15.16:0/4108383112 conn(0x7f3cec099b40 msgr2=0x7f3cec06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 -- 172.21.15.16:0/4108383112 shutdown_connections 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.612+0000 7f3cf39ba700 1 -- 172.21.15.16:0/4108383112 wait complete. 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.613+0000 7f3cf39ba700 1 Processor -- start 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.613+0000 7f3cf39ba700 1 -- start start 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0cf450 0x7f3cec0ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec15cc20 0x7f3cec0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b45e0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b3cb0 con 0x7f3cec0cf450 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf39ba700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b3f90 con 0x7f3cec15cc20 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf1756700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf1756700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57230/0 (socket says 172.21.15.16:57230) 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf1756700 1 -- 172.21.15.16:0/1142853523 learned_addr learned my addr 172.21.15.16:0/1142853523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf1f57700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec15cc20 0x7f3cec0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.614+0000 7f3cf0f55700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0cf450 0x7f3cec0ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 -- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec15cc20 msgr2=0x7f3cec0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec15cc20 0x7f3cec0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 -- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0cf450 msgr2=0x7f3cec0ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0cf450 0x7f3cec0ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 -- 172.21.15.16:0/1142853523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80094f0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf1756700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cec089390 tx=0x7f3ce8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce801f070 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce8011ae0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec0b4270 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8002f70 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.615+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec0caa80 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.616+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec1548f0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.616+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f3ce8023070 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.617+0000 7f3ce17fa700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 0x7f3cd0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.617+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f3ce8085560 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.617+0000 7f3cf0f55700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 0x7f3cd0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.617+0000 7f3cf0f55700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 0x7f3cd0069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0b3990 tx=0x7f3cdc007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.621+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ce8052ab0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.817+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cec04c380 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.818+0000 7f3ce17fa700 1 -- 172.21.15.16:0/1142853523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13666 (secure 0 0 0) 0x7f3ce80560d0 con 0x7f3cec0cd6a0 2023-10-06T11:41:50.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:50.820 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:50.444838+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 msgr2=0x7f3cd0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 0x7f3cd0069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0b3990 tx=0x7f3cdc007ec0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 msgr2=0x7f3cec1543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cec089390 tx=0x7f3ce8017000 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 shutdown_connections 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0cd6a0 0x7f3cec1543b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd0066ea0 0x7f3cd0069350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec15cc20 0x7f3cec0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 --2- 172.21.15.16:0/1142853523 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0cf450 0x7f3cec0ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 >> 172.21.15.16:0/1142853523 conn(0x7f3cec099b40 msgr2=0x7f3cec06f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 shutdown_connections 2023-10-06T11:41:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.820+0000 7f3cf39ba700 1 -- 172.21.15.16:0/1142853523 wait complete. 2023-10-06T11:41:50.834 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2023-10-06T11:41:50.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:41:50.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.955+0000 7f2b98b9d700 1 Processor -- start 2023-10-06T11:41:50.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.955+0000 7f2b98b9d700 1 -- start start 2023-10-06T11:41:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.955+0000 7f2b98b9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b94159740 0x7f2b940b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.955+0000 7f2b98b9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b940cda10 0x7f2b940b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.955+0000 7f2b98b9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 0x7f2b940b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b98b9d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940cead0 con 0x7f2b940cda10 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b98b9d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940affc0 con 0x7f2b94159b10 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b98b9d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940cdec0 con 0x7f2b94159740 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b940cda10 0x7f2b940b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b940cda10 0x7f2b940b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57232/0 (socket says 172.21.15.16:57232) 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 -- 172.21.15.16:0/1114550922 learned_addr learned my addr 172.21.15.16:0/1114550922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b92d9d700 1 --2- 172.21.15.16:0/1114550922 >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 0x7f2b940b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b9259c700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b94159740 0x7f2b940b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 -- 172.21.15.16:0/1114550922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b94159740 msgr2=0x7f2b940b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b94159740 0x7f2b940b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 -- 172.21.15.16:0/1114550922 >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 msgr2=0x7f2b940b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:50.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/1114550922 >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 0x7f2b940b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 -- 172.21.15.16:0/1114550922 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940ce1a0 con 0x7f2b940cda10 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b92d9d700 1 --2- 172.21.15.16:0/1114550922 >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 0x7f2b940b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b940cda10 0x7f2b940b3ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b940b3240 tx=0x7f2b7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b13c0f02fac507 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b9159a700 1 -- 172.21.15.16:0/1114550922 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00b860 con 0x7f2b940cda10 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b9159a700 1 -- 172.21.15.16:0/1114550922 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c00b9c0 con 0x7f2b940cda10 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.956+0000 7f2b9159a700 1 -- 172.21.15.16:0/1114550922 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c00bd60 con 0x7f2b940cda10 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 -- 172.21.15.16:0/1114550922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b940cda10 msgr2=0x7f2b940b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b940cda10 0x7f2b940b3ed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b940b3240 tx=0x7f2b7c009250 comp rx=0 tx=0).stop 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 -- 172.21.15.16:0/1114550922 shutdown_connections 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/1114550922 >> v2:172.21.15.177:3300/0 conn(0x7f2b94159b10 0x7f2b940b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b94159740 0x7f2b940b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/1114550922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b940cda10 0x7f2b940b3ed0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 -- 172.21.15.16:0/1114550922 >> 172.21.15.16:0/1114550922 conn(0x7f2b94099a40 msgr2=0x7f2b9406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 -- 172.21.15.16:0/1114550922 shutdown_connections 2023-10-06T11:41:50.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.957+0000 7f2b98b9d700 1 -- 172.21.15.16:0/1114550922 wait complete. 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.958+0000 7f2b98b9d700 1 Processor -- start 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.958+0000 7f2b98b9d700 1 -- start start 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b940cda10 0x7f2b94147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b94159b10 0x7f2b9415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9409ad80 con 0x7f2b94159740 2023-10-06T11:41:50.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9415a280 con 0x7f2b94159b10 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9409a450 con 0x7f2b940cda10 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57236/0 (socket says 172.21.15.16:57236) 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 -- 172.21.15.16:0/3693958453 learned_addr learned my addr 172.21.15.16:0/3693958453 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b9259c700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b940cda10 0x7f2b94147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b92d9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b94159b10 0x7f2b9415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 -- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b940cda10 msgr2=0x7f2b94147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b940cda10 0x7f2b94147a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 -- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b94159b10 msgr2=0x7f2b9415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b94159b10 0x7f2b9415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 -- 172.21.15.16:0/3693958453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0094f0 con 0x7f2b94159740 2023-10-06T11:41:50.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b91d9b700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009290 tx=0x7f2b7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c011e40 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c01a450 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c01a810 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.959+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9409a730 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.960+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b94147fc0 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.960+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9409aef0 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.961+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2b7c01abc0 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.961+0000 7f2b8a7fc700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 0x7f2b74069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.961+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f2b7c0857f0 con 0x7f2b94159740 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.961+0000 7f2b9259c700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 0x7f2b74069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:50.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.962+0000 7f2b9259c700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 0x7f2b74069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007640 tx=0x7f2b84009850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:50.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:50.966+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b7c052d40 con 0x7f2b94159740 2023-10-06T11:41:51.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.156+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2b9404c380 con 0x7f2b94159740 2023-10-06T11:41:51.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.156+0000 7f2b8a7fc700 1 -- 172.21.15.16:0/3693958453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2b7c056360 con 0x7f2b94159740 2023-10-06T11:41:51.159 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:41:51.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 msgr2=0x7f2b74069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 0x7f2b74069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2b84007640 tx=0x7f2b84009850 comp rx=0 tx=0).stop 2023-10-06T11:41:51.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 msgr2=0x7f2b9415ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009290 tx=0x7f2b7c017000 comp rx=0 tx=0).stop 2023-10-06T11:41:51.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 shutdown_connections 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b74066ea0 0x7f2b74069350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b94159b10 0x7f2b9415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b940cda10 0x7f2b94147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 --2- 172.21.15.16:0/3693958453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b94159740 0x7f2b9415ae70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 >> 172.21.15.16:0/3693958453 conn(0x7f2b94099a40 msgr2=0x7f2b94158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 shutdown_connections 2023-10-06T11:41:51.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.159+0000 7f2b98b9d700 1 -- 172.21.15.16:0/3693958453 wait complete. 2023-10-06T11:41:51.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:41:51.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.294+0000 7fc718517700 1 Processor -- start 2023-10-06T11:41:51.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.294+0000 7fc718517700 1 -- start start 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc718517700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 0x7fc7100caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc718517700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71015e210 con 0x7fc710143100 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc718517700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71006d520 con 0x7fc710142d30 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc718517700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71006e0a0 con 0x7fc7100cc7d0 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc715ab2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc710143100 0x7fc7100cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46126/0 (socket says 172.21.15.16:46126) 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc715ab2700 1 -- 172.21.15.16:0/3264160812 learned_addr learned my addr 172.21.15.16:0/3264160812 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.295+0000 7fc716ab4700 1 --2- 172.21.15.16:0/3264160812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 0x7fc7100caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735699953 0 0) 0x7fc71015e210 con 0x7fc710143100 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6f8003570 con 0x7fc710143100 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1000395901 0 0) 0x7fc71006d520 con 0x7fc710142d30 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71015e210 con 0x7fc710142d30 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc716ab4700 1 -- 172.21.15.16:0/3264160812 >> v1:172.21.15.177:6789/0 conn(0x7fc710142d30 legacy=0x7fc71006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc716ab4700 1 -- 172.21.15.16:0/3264160812 >> v1:172.21.15.16:6789/0 conn(0x7fc710143100 legacy=0x7fc7100cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc716ab4700 1 -- 172.21.15.16:0/3264160812 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71006d770 con 0x7fc7100cc7d0 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc716ab4700 1 --2- 172.21.15.16:0/3264160812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 0x7fc7100caef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc708009910 tx=0x7fc7080091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bb0d7707f928d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708011630 con 0x7fc7100cc7d0 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708011790 con 0x7fc7100cc7d0 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.296+0000 7fc7152b1700 1 -- 172.21.15.16:0/3264160812 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708011b10 con 0x7fc7100cc7d0 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 -- 172.21.15.16:0/3264160812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 msgr2=0x7fc7100caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 --2- 172.21.15.16:0/3264160812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 0x7fc7100caef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc708009910 tx=0x7fc7080091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 -- 172.21.15.16:0/3264160812 shutdown_connections 2023-10-06T11:41:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 --2- 172.21.15.16:0/3264160812 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7100cc7d0 0x7fc7100caef0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 -- 172.21.15.16:0/3264160812 >> 172.21.15.16:0/3264160812 conn(0x7fc71006cee0 msgr2=0x7fc710067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 -- 172.21.15.16:0/3264160812 shutdown_connections 2023-10-06T11:41:51.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.297+0000 7fc718517700 1 -- 172.21.15.16:0/3264160812 wait complete. 2023-10-06T11:41:51.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.298+0000 7fc718517700 1 Processor -- start 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.298+0000 7fc718517700 1 -- start start 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc710142d30 0x7fc71016cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc710143100 0x7fc710169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71016d920 con 0x7fc7100cc7d0 2023-10-06T11:41:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71016dc00 con 0x7fc710143100 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7100c5740 con 0x7fc710142d30 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57248/0 (socket says 172.21.15.16:57248) 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 -- 172.21.15.16:0/1532404496 learned_addr learned my addr 172.21.15.16:0/1532404496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc716ab4700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc710143100 0x7fc710169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc715ab2700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc710142d30 0x7fc71016cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 -- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc710142d30 msgr2=0x7fc71016cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc710142d30 0x7fc71016cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 -- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc710143100 msgr2=0x7fc710169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc710143100 0x7fc710169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 -- 172.21.15.16:0/1532404496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc708009420 con 0x7fc7100cc7d0 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc7162b3700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc70000cb40 tx=0x7fc7000027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7000040f0 con 0x7fc7100cc7d0 2023-10-06T11:41:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc70000a7e0 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7100c5a20 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70000aba0 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.299+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71016d4f0 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.300+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7100c5580 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.301+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc700012420 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.301+0000 7fc70e7fc700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 0x7fc6fc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.301+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc700084220 con 0x7fc7100cc7d0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.301+0000 7fc715ab2700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 0x7fc6fc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.302+0000 7fc715ab2700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 0x7fc6fc069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b2dd0 tx=0x7fc704009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.305+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc700051830 con 0x7fc7100cc7d0 2023-10-06T11:41:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.506+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc71004c380 con 0x7fc7100cc7d0 2023-10-06T11:41:51.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.507+0000 7fc70e7fc700 1 -- 172.21.15.16:0/1532404496 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc700054e50 con 0x7fc7100cc7d0 2023-10-06T11:41:51.510 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 msgr2=0x7fc6fc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 0x7fc6fc069350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc7100b2dd0 tx=0x7fc704009040 comp rx=0 tx=0).stop 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 msgr2=0x7fc71013d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc70000cb40 tx=0x7fc7000027a0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 shutdown_connections 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc7100cc7d0 0x7fc71013d860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc6fc066ea0 0x7fc6fc069350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc710143100 0x7fc710169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 --2- 172.21.15.16:0/1532404496 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc710142d30 0x7fc71016cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 >> 172.21.15.16:0/1532404496 conn(0x7fc71006cee0 msgr2=0x7fc710069330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 shutdown_connections 2023-10-06T11:41:51.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.509+0000 7fc718517700 1 -- 172.21.15.16:0/1532404496 wait complete. 2023-10-06T11:41:51.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:41:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.644+0000 7f4d21164700 1 Processor -- start 2023-10-06T11:41:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.644+0000 7f4d21164700 1 -- start start 2023-10-06T11:41:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d21164700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4d1c0c65a0 0x7f4d1c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d21164700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0ccb20 con 0x7f4d1c0cd870 2023-10-06T11:41:51.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d21164700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0cbf10 con 0x7f4d1c15cc20 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d21164700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0b3110 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d1c0cd870 0x7f4d1c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46142/0 (socket says 172.21.15.16:46142) 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1259c700 1 -- 172.21.15.16:0/2780602818 learned_addr learned my addr 172.21.15.16:0/2780602818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1b59e700 1 --2- 172.21.15.16:0/2780602818 >> v2:172.21.15.177:3300/0 conn(0x7f4d1c0c65a0 0x7f4d1c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300633125 0 0) 0x7f4d1c0ccb20 con 0x7f4d1c0cd870 2023-10-06T11:41:51.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cfc003570 con 0x7f4d1c0cd870 2023-10-06T11:41:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086839567 0 0) 0x7f4d1c0cbf10 con 0x7f4d1c15cc20 2023-10-06T11:41:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.645+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d1c0ccb20 con 0x7f4d1c15cc20 2023-10-06T11:41:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1b59e700 1 -- 172.21.15.16:0/2780602818 >> v1:172.21.15.177:6789/0 conn(0x7f4d1c15cc20 legacy=0x7f4d1c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1b59e700 1 -- 172.21.15.16:0/2780602818 >> v1:172.21.15.16:6789/0 conn(0x7f4d1c0cd870 legacy=0x7f4d1c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1b59e700 1 -- 172.21.15.16:0/2780602818 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0cc1f0 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1b59e700 1 --2- 172.21.15.16:0/2780602818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0cf450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009e60 tx=0x7f4d14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faef0483586a6c0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e070 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400bab0 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d1a59c700 1 -- 172.21.15.16:0/2780602818 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400be50 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d21164700 1 -- 172.21.15.16:0/2780602818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 msgr2=0x7f4d1c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.646+0000 7f4d21164700 1 --2- 172.21.15.16:0/2780602818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0cf450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009e60 tx=0x7f4d14009780 comp rx=0 tx=0).stop 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.647+0000 7f4d21164700 1 -- 172.21.15.16:0/2780602818 shutdown_connections 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.647+0000 7f4d21164700 1 --2- 172.21.15.16:0/2780602818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0cf450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.647+0000 7f4d21164700 1 -- 172.21.15.16:0/2780602818 >> 172.21.15.16:0/2780602818 conn(0x7f4d1c099b40 msgr2=0x7f4d1c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.647+0000 7f4d21164700 1 -- 172.21.15.16:0/2780602818 shutdown_connections 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.647+0000 7f4d21164700 1 -- 172.21.15.16:0/2780602818 wait complete. 2023-10-06T11:41:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.648+0000 7f4d21164700 1 Processor -- start 2023-10-06T11:41:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.648+0000 7f4d21164700 1 -- start start 2023-10-06T11:41:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d1c0cd870 0x7f4d1c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c7ac0 con 0x7f4d1c15cc20 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c169fe0 con 0x7f4d1c0c65a0 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d21164700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c16a2c0 con 0x7f4d1c0cd870 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57254/0 (socket says 172.21.15.16:57254) 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 -- 172.21.15.16:0/3463532134 learned_addr learned my addr 172.21.15.16:0/3463532134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:51.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1259c700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d1c0cd870 0x7f4d1c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1ad9d700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 -- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d1c0cd870 msgr2=0x7f4d1c1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d1c0cd870 0x7f4d1c1542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 -- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 msgr2=0x7f4d1c0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.649+0000 7f4d1b59e700 1 -- 172.21.15.16:0/3463532134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009a20 con 0x7f4d1c15cc20 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d1b59e700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c089390 tx=0x7f4d14019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14007080 con 0x7f4d1c15cc20 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14012d00 con 0x7f4d1c15cc20 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c0c94e0 con 0x7f4d1c15cc20 2023-10-06T11:41:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14026640 con 0x7f4d1c15cc20 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.650+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c0b3a40 con 0x7f4d1c15cc20 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.651+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c0c9a00 con 0x7f4d1c15cc20 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.651+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f4d14026a70 con 0x7f4d1c15cc20 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.652+0000 7f4d137fe700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 0x7f4d00069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.652+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4d14085f90 con 0x7f4d1c15cc20 2023-10-06T11:41:51.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.652+0000 7f4d1ad9d700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 0x7f4d00069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.652+0000 7f4d1ad9d700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 0x7f4d00069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d04007fd0 tx=0x7f4d04007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:51.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.656+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d1c0c9a00 con 0x7f4d1c15cc20 2023-10-06T11:41:51.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.905+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1c0c9a00 con 0x7f4d1c15cc20 2023-10-06T11:41:51.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.905+0000 7f4d137fe700 1 -- 172.21.15.16:0/3463532134 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f4d1c0c9a00 con 0x7f4d1c15cc20 2023-10-06T11:41:51.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:51.907 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4297,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T11:41:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 msgr2=0x7f4d00069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 0x7f4d00069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d04007fd0 tx=0x7f4d04007d50 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 msgr2=0x7f4d1c169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c089390 tx=0x7f4d14019000 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 shutdown_connections 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d1c15cc20 0x7f4d1c169420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4d00066ea0 0x7f4d00069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4d1c0c65a0 0x7f4d1c0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 --2- 172.21.15.16:0/3463532134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4d1c0cd870 0x7f4d1c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:51.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 >> 172.21.15.16:0/3463532134 conn(0x7f4d1c099b40 msgr2=0x7f4d1c0c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 shutdown_connections 2023-10-06T11:41:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:51.907+0000 7f4d21164700 1 -- 172.21.15.16:0/3463532134 wait complete. 2023-10-06T11:41:51.911 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2023-10-06T11:41:51.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.039+0000 7f49b5a41700 1 Processor -- start 2023-10-06T11:41:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.039+0000 7f49b5a41700 1 -- start start 2023-10-06T11:41:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49b5a41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b01540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49b5a41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49b006c410 0x7f49b006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49b5a41700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b006cdc0 con 0x7f49b00675c0 2023-10-06T11:41:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49b5a41700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b0067a30 con 0x7f49b006c410 2023-10-06T11:41:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49b5a41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b015e4f0 con 0x7f49b0153cd0 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.040+0000 7f49af7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49b006c410 0x7f49b006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49b006c410 0x7f49b006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57256/0 (socket says 172.21.15.16:57256) 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 -- 172.21.15.16:0/499485395 learned_addr learned my addr 172.21.15.16:0/499485395 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49aeffd700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b01540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 -- 172.21.15.16:0/499485395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 msgr2=0x7f49b01540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b01540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 -- 172.21.15.16:0/499485395 >> v1:172.21.15.177:6789/0 conn(0x7f49b00675c0 legacy=0x7f49b00af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 -- 172.21.15.16:0/499485395 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b015d600 con 0x7f49b006c410 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49af7fe700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b006c880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49a80099b0 tx=0x7f49a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fc3b83215eefc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49adffb700 1 -- 172.21.15.16:0/499485395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8009ea0 con 0x7f49b006c410 2023-10-06T11:41:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49adffb700 1 -- 172.21.15.16:0/499485395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a800b860 con 0x7f49b006c410 2023-10-06T11:41:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.041+0000 7f49adffb700 1 -- 172.21.15.16:0/499485395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a800bc00 con 0x7f49b006c410 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 -- 172.21.15.16:0/499485395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 msgr2=0x7f49b006c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b006c880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49a80099b0 tx=0x7f49a8009250 comp rx=0 tx=0).stop 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 -- 172.21.15.16:0/499485395 shutdown_connections 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b006c880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 --2- 172.21.15.16:0/499485395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b01540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 -- 172.21.15.16:0/499485395 >> 172.21.15.16:0/499485395 conn(0x7f49b00cd480 msgr2=0x7f49b00cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 -- 172.21.15.16:0/499485395 shutdown_connections 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.042+0000 7f49b5a41700 1 -- 172.21.15.16:0/499485395 wait complete. 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.043+0000 7f49b5a41700 1 Processor -- start 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.043+0000 7f49b5a41700 1 -- start start 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b00675c0 0x7f49b016a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b0159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b015bb00 con 0x7f49b006c410 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b016ac20 con 0x7f49b00675c0 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49b5a41700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b014ee30 con 0x7f49b0153cd0 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49af7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b0159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57258/0 (socket says 172.21.15.16:57258) 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49aeffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b00675c0 0x7f49b016a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 -- 172.21.15.16:0/4290284180 learned_addr learned my addr 172.21.15.16:0/4290284180 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:52.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 -- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 msgr2=0x7f49b0159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b0159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 -- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b00675c0 msgr2=0x7f49b016a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b00675c0 0x7f49b016a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 -- 172.21.15.16:0/4290284180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80094f0 con 0x7f49b006c410 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.044+0000 7f49ae7fc700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f499800c4e0 tx=0x7f499800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998011590 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4998011710 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b014f340 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4998011ad0 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b015b250 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.045+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b004c380 con 0x7f49b006c410 2023-10-06T11:41:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.046+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f499801b3b0 con 0x7f49b006c410 2023-10-06T11:41:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.046+0000 7f49a6ffd700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 0x7f4990069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.047+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4998083650 con 0x7f49b006c410 2023-10-06T11:41:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.047+0000 7f49aeffd700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 0x7f4990069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.047+0000 7f49aeffd700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 0x7f4990069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a000b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.051+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4998050af0 con 0x7f49b006c410 2023-10-06T11:41:52.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.247+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49b00afd50 con 0x7f49b006c410 2023-10-06T11:41:52.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.248+0000 7f49a6ffd700 1 -- 172.21.15.16:0/4290284180 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13661 (secure 0 0 0) 0x7f4998017070 con 0x7f49b006c410 2023-10-06T11:41:52.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:52.250 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:51.451035+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 msgr2=0x7f4990069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 0x7f4990069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009fd0 tx=0x7f49a000b040 comp rx=0 tx=0).stop 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 msgr2=0x7f49b014e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f499800c4e0 tx=0x7f499800c7f0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 shutdown_connections 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4990066ea0 0x7f4990069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49b0153cd0 0x7f49b0159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49b00675c0 0x7f49b016a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.250+0000 7f49b5a41700 1 --2- 172.21.15.16:0/4290284180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b006c410 0x7f49b014e8f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.251+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 >> 172.21.15.16:0/4290284180 conn(0x7f49b00cd480 msgr2=0x7f49b00cb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.251+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 shutdown_connections 2023-10-06T11:41:52.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.251+0000 7f49b5a41700 1 -- 172.21.15.16:0/4290284180 wait complete. 2023-10-06T11:41:52.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T11:41:52.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:41:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.387+0000 7f88ddff3700 1 Processor -- start 2023-10-06T11:41:52.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.387+0000 7f88ddff3700 1 -- start start 2023-10-06T11:41:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88d8159740 0x7f88d80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d80b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 0x7f88d80b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d80cead0 con 0x7f88d8159b10 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d80affc0 con 0x7f88d8159740 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88ddff3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d80cdec0 con 0x7f88d80cda10 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 0x7f88d80b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d80b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88d8159740 0x7f88d80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88d8159740 0x7f88d80b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:49234/0 (socket says 172.21.15.16:49234) 2023-10-06T11:41:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d80b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41946/0 (socket says 172.21.15.16:41946) 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 -- 172.21.15.16:0/382371878 learned_addr learned my addr 172.21.15.16:0/382371878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d7fff700 1 --2- 172.21.15.16:0/382371878 >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 0x7f88d80b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57264/0 (socket says 172.21.15.16:57264) 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 -- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 msgr2=0x7f88d80b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 --2- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d80b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 -- 172.21.15.16:0/382371878 >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 msgr2=0x7f88d80b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 --2- 172.21.15.16:0/382371878 >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 0x7f88d80b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.388+0000 7f88d77fe700 1 -- 172.21.15.16:0/382371878 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80ce1a0 con 0x7f88d8159740 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88d77fe700 1 --2- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159740 0x7f88d80b3990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88d8089030 tx=0x7f88c0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d60f0a4992c6bac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88d67fc700 1 -- 172.21.15.16:0/382371878 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000b640 con 0x7f88d8159740 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88d67fc700 1 -- 172.21.15.16:0/382371878 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c000b7a0 con 0x7f88d8159740 2023-10-06T11:41:52.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88d67fc700 1 -- 172.21.15.16:0/382371878 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000bb60 con 0x7f88d8159740 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 -- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159740 msgr2=0x7f88d80b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 --2- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159740 0x7f88d80b3990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f88d8089030 tx=0x7f88c0009250 comp rx=0 tx=0).stop 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 -- 172.21.15.16:0/382371878 shutdown_connections 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 --2- 172.21.15.16:0/382371878 >> v2:172.21.15.16:3300/0 conn(0x7f88d8159b10 0x7f88d80b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 --2- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159740 0x7f88d80b3990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 --2- 172.21.15.16:0/382371878 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d80b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 -- 172.21.15.16:0/382371878 >> 172.21.15.16:0/382371878 conn(0x7f88d8099a40 msgr2=0x7f88d806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 -- 172.21.15.16:0/382371878 shutdown_connections 2023-10-06T11:41:52.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.389+0000 7f88ddff3700 1 -- 172.21.15.16:0/382371878 wait complete. 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 Processor -- start 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 -- start start 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 0x7f88d80b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d815a300 con 0x7f88d8159740 2023-10-06T11:41:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d809de50 con 0x7f88d8159b10 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88ddff3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d809e130 con 0x7f88d80cda10 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88d6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57278/0 (socket says 172.21.15.16:57278) 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.391+0000 7f88d6ffd700 1 -- 172.21.15.16:0/3357159674 learned_addr learned my addr 172.21.15.16:0/3357159674 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d77fe700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d7fff700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 0x7f88d80b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 -- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 msgr2=0x7f88d8156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d8156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 -- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 msgr2=0x7f88d80b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 0x7f88d80b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 -- 172.21.15.16:0/3357159674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00094f0 con 0x7f88d8159740 2023-10-06T11:41:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d7fff700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 0x7f88d80b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.392+0000 7f88d6ffd700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88c800ec80 tx=0x7f88c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.405+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800c9d0 con 0x7f88d8159740 2023-10-06T11:41:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.405+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c800cb30 con 0x7f88d8159740 2023-10-06T11:41:52.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.405+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d809acc0 con 0x7f88d8159740 2023-10-06T11:41:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.405+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8013470 con 0x7f88d8159740 2023-10-06T11:41:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.405+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d81570d0 con 0x7f88d8159740 2023-10-06T11:41:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.406+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d809b160 con 0x7f88d8159740 2023-10-06T11:41:52.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.406+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f88c8013820 con 0x7f88d8159740 2023-10-06T11:41:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.407+0000 7f88cf7fe700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 0x7f88b8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.407+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f88c8084650 con 0x7f88d8159740 2023-10-06T11:41:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.407+0000 7f88d77fe700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 0x7f88b8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.407+0000 7f88d77fe700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 0x7f88b8069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88d8089030 tx=0x7f88c0017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.413+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88c8051af0 con 0x7f88d8159740 2023-10-06T11:41:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.607+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f88d809ab20 con 0x7f88d8159740 2023-10-06T11:41:52.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.607+0000 7f88cf7fe700 1 -- 172.21.15.16:0/3357159674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f88d809ab20 con 0x7f88d8159740 2023-10-06T11:41:52.610 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 msgr2=0x7f88b8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 0x7f88b8069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88d8089030 tx=0x7f88c0017000 comp rx=0 tx=0).stop 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 msgr2=0x7f88d809b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88c800ec80 tx=0x7f88c80091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 shutdown_connections 2023-10-06T11:41:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f88b8066ea0 0x7f88b8069350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88d8159b10 0x7f88d80b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88d80cda10 0x7f88d8156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 --2- 172.21.15.16:0/3357159674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88d8159740 0x7f88d809b4a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 >> 172.21.15.16:0/3357159674 conn(0x7f88d8099a40 msgr2=0x7f88d80cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 shutdown_connections 2023-10-06T11:41:52.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.609+0000 7f88ddff3700 1 -- 172.21.15.16:0/3357159674 wait complete. 2023-10-06T11:41:52.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:41:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.743+0000 7f897d4d3700 1 Processor -- start 2023-10-06T11:41:52.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.743+0000 7f897d4d3700 1 -- start start 2023-10-06T11:41:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f897d4d3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f89780cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f897d4d3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89780cc600 0x7f89780caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f897d4d3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897815e210 con 0x7f8978142d30 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f897d4d3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780cca60 con 0x7f89780cc600 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f897d4d3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897806d520 con 0x7f8978143100 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89767fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f89780cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f8976ffd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8978142d30 0x7f897806e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:60390/0 (socket says 172.21.15.16:60390) 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f89780cc600 0x7f89780caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f8976ffd700 1 -- 172.21.15.16:0/1716906271 learned_addr learned my addr 172.21.15.16:0/1716906271 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f8975ffb700 1 -- 172.21.15.16:0/1716906271 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400059553 0 0) 0x7f897815e210 con 0x7f8978142d30 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f8975ffb700 1 -- 172.21.15.16:0/1716906271 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8964003570 con 0x7f8978142d30 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 -- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 msgr2=0x7f89780cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f89780cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 -- 172.21.15.16:0/1716906271 >> v1:172.21.15.177:6789/0 conn(0x7f8978142d30 legacy=0x7f897806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 -- 172.21.15.16:0/1716906271 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897806e0a0 con 0x7f89780cc600 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89767fc700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f89780cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.744+0000 7f89777fe700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780caef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89700098e0 tx=0x7f89700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4a6d6e438492b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f8975ffb700 1 -- 172.21.15.16:0/1716906271 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970011630 con 0x7f89780cc600 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f8975ffb700 1 -- 172.21.15.16:0/1716906271 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8970011790 con 0x7f89780cc600 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f8975ffb700 1 -- 172.21.15.16:0/1716906271 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970011b30 con 0x7f89780cc600 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 -- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 msgr2=0x7f89780caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780caef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89700098e0 tx=0x7f89700091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 -- 172.21.15.16:0/1716906271 shutdown_connections 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780caef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1716906271 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f89780cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 -- 172.21.15.16:0/1716906271 >> 172.21.15.16:0/1716906271 conn(0x7f897806cee0 msgr2=0x7f8978067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 -- 172.21.15.16:0/1716906271 shutdown_connections 2023-10-06T11:41:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.745+0000 7f897d4d3700 1 -- 172.21.15.16:0/1716906271 wait complete. 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.746+0000 7f897d4d3700 1 Processor -- start 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.746+0000 7f897d4d3700 1 -- start start 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f8978159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89780b2ac0 con 0x7f8978142d30 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89780b1c10 con 0x7f89780cc600 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f897d4d3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89780b1eb0 con 0x7f8978143100 2023-10-06T11:41:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f89767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f89767fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57284/0 (socket says 172.21.15.16:57284) 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f89767fc700 1 -- 172.21.15.16:0/1741205664 learned_addr learned my addr 172.21.15.16:0/1741205664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f89777fe700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f8978159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.747+0000 7f8976ffd700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 -- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 msgr2=0x7f8978159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f8978159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 -- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 msgr2=0x7f89780b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 -- 172.21.15.16:0/1741205664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960009420 con 0x7f8978142d30 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f8976ffd700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89777fe700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f8978159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f89767fc700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f896000ec80 tx=0x7f89600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896000c9d0 con 0x7f8978142d30 2023-10-06T11:41:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896000cb30 con 0x7f8978142d30 2023-10-06T11:41:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8970009420 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896000ceb0 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.748+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89780b2470 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.749+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89780b2c30 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.749+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f8960013680 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.750+0000 7f896effd700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 0x7f8958069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.750+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8960084670 con 0x7f8978142d30 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.750+0000 7f8976ffd700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 0x7f8958069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.750+0000 7f8976ffd700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 0x7f8958069350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8968005c50 tx=0x7f8968009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.754+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8960051bb0 con 0x7f8978142d30 2023-10-06T11:41:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.947+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f897804c380 con 0x7f8978142d30 2023-10-06T11:41:52.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.947+0000 7f896effd700 1 -- 172.21.15.16:0/1741205664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89600551d0 con 0x7f8978142d30 2023-10-06T11:41:52.950 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:41:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 msgr2=0x7f8958069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 0x7f8958069350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8968005c50 tx=0x7f8968009040 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 msgr2=0x7f897815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f896000ec80 tx=0x7f89600091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 shutdown_connections 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8958066ea0 0x7f8958069350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8978143100 0x7f8978159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89780cc600 0x7f89780b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 --2- 172.21.15.16:0/1741205664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8978142d30 0x7f897815c9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 >> 172.21.15.16:0/1741205664 conn(0x7f897806cee0 msgr2=0x7f897813fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 shutdown_connections 2023-10-06T11:41:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:52.949+0000 7f897d4d3700 1 -- 172.21.15.16:0/1741205664 wait complete. 2023-10-06T11:41:52.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:41:53.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.079+0000 7f984817b700 1 Processor -- start 2023-10-06T11:41:53.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.080+0000 7f984817b700 1 -- start start 2023-10-06T11:41:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.080+0000 7f984817b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f984015cc20 0x7f98400cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.080+0000 7f984817b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400cdb10 con 0x7f98400cd6a0 2023-10-06T11:41:53.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.080+0000 7f984817b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98400ccb20 con 0x7f98400cf450 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.080+0000 7f984817b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98400b3110 con 0x7f984015cc20 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845716700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98400cd6a0 0x7f98400ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46156/0 (socket says 172.21.15.16:46156) 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845716700 1 -- 172.21.15.16:0/993183457 learned_addr learned my addr 172.21.15.16:0/993183457 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845f17700 1 --2- 172.21.15.16:0/993183457 >> v2:172.21.15.177:3300/0 conn(0x7f984015cc20 0x7f98400cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372902208 0 0) 0x7f98400cdb10 con 0x7f98400cd6a0 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f982c003570 con 0x7f98400cd6a0 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845f17700 1 -- 172.21.15.16:0/993183457 >> v1:172.21.15.177:6789/0 conn(0x7f98400cf450 legacy=0x7f98400c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303193223 0 0) 0x7f98400ccb20 con 0x7f98400cf450 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845f17700 1 -- 172.21.15.16:0/993183457 >> v1:172.21.15.16:6789/0 conn(0x7f98400cd6a0 legacy=0x7f98400ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845f17700 1 -- 172.21.15.16:0/993183457 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400cf880 con 0x7f984015cc20 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9845f17700 1 --2- 172.21.15.16:0/993183457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f98400cd160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f983c0098e0 tx=0x7f983c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96648aacec40254 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00b640 con 0x7f984015cc20 2023-10-06T11:41:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00b7a0 con 0x7f984015cc20 2023-10-06T11:41:53.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.081+0000 7f9844f15700 1 -- 172.21.15.16:0/993183457 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00bb40 con 0x7f984015cc20 2023-10-06T11:41:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 -- 172.21.15.16:0/993183457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 msgr2=0x7f98400cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 --2- 172.21.15.16:0/993183457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f98400cd160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f983c0098e0 tx=0x7f983c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 -- 172.21.15.16:0/993183457 shutdown_connections 2023-10-06T11:41:53.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 --2- 172.21.15.16:0/993183457 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f98400cd160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 -- 172.21.15.16:0/993183457 >> 172.21.15.16:0/993183457 conn(0x7f9840099b40 msgr2=0x7f984006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 -- 172.21.15.16:0/993183457 shutdown_connections 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.082+0000 7f984817b700 1 -- 172.21.15.16:0/993183457 wait complete. 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.083+0000 7f984817b700 1 Processor -- start 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.083+0000 7f984817b700 1 -- start start 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98400cf450 0x7f98400b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f984015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840156c80 con 0x7f98400cd6a0 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984015e290 con 0x7f984015cc20 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f984817b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984015e570 con 0x7f98400cf450 2023-10-06T11:41:53.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57288/0 (socket says 172.21.15.16:57288) 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 -- 172.21.15.16:0/579083154 learned_addr learned my addr 172.21.15.16:0/579083154 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9846718700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f984015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845716700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98400cf450 0x7f98400b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 -- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98400cf450 msgr2=0x7f98400b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98400cf450 0x7f98400b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 -- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 msgr2=0x7f984015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f984015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 -- 172.21.15.16:0/579083154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c009420 con 0x7f98400cd6a0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9845f17700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9840089390 tx=0x7f983c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.084+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c01f070 con 0x7f98400cd6a0 2023-10-06T11:41:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.085+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c002f70 con 0x7f98400cd6a0 2023-10-06T11:41:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.085+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984016a530 con 0x7f98400cd6a0 2023-10-06T11:41:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.085+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00b910 con 0x7f98400cd6a0 2023-10-06T11:41:53.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.085+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984016aaf0 con 0x7f98400cd6a0 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.086+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984015e850 con 0x7f98400cd6a0 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.086+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f983c023070 con 0x7f98400cd6a0 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.086+0000 7f9835ffb700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 0x7f9824069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.086+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f983c085490 con 0x7f98400cd6a0 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.087+0000 7f9845716700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 0x7f9824069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.087+0000 7f9845716700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 0x7f9824069350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9840156300 tx=0x7f9830009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.091+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f983c0529d0 con 0x7f98400cd6a0 2023-10-06T11:41:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.101+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f983c052da0 con 0x7f98400cd6a0 2023-10-06T11:41:53.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.287+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f984004c380 con 0x7f98400cd6a0 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.288+0000 7f9835ffb700 1 -- 172.21.15.16:0/579083154 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f983c056460 con 0x7f98400cd6a0 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-10-06T11:41:53.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:41:53.291 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:41:53.291 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-10-06T11:41:53.291 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-10-06T11:41:53.291 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:41:53.291 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-10-06T11:41:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 msgr2=0x7f9824069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 0x7f9824069350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9840156300 tx=0x7f9830009040 comp rx=0 tx=0).stop 2023-10-06T11:41:53.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 msgr2=0x7f9840169d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9840089390 tx=0x7f983c017000 comp rx=0 tx=0).stop 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 shutdown_connections 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98400cd6a0 0x7f9840169d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9824066ea0 0x7f9824069350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f984015cc20 0x7f984015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 --2- 172.21.15.16:0/579083154 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f98400cf450 0x7f98400b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 >> 172.21.15.16:0/579083154 conn(0x7f9840099b40 msgr2=0x7f984006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 shutdown_connections 2023-10-06T11:41:53.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.290+0000 7f984817b700 1 -- 172.21.15.16:0/579083154 wait complete. 2023-10-06T11:41:53.304 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:41:53.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:41:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.424+0000 7f7879914700 1 Processor -- start 2023-10-06T11:41:53.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.424+0000 7f7879914700 1 -- start start 2023-10-06T11:41:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7879914700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7874159810 0x7f78740ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7879914700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f78740b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7879914700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740b4960 con 0x7f78740b6170 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7879914700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740b31d0 con 0x7f7874159810 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7879914700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b2630 con 0x7f78740b3fc0 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7874159810 0x7f78740ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7874159810 0x7f78740ab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57296/0 (socket says 172.21.15.16:57296) 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 -- 172.21.15.16:0/4128804921 learned_addr learned my addr 172.21.15.16:0/4128804921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f78737fe700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f78740b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 -- 172.21.15.16:0/4128804921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 msgr2=0x7f78740b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f78740b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 -- 172.21.15.16:0/4128804921 >> v1:172.21.15.177:6789/0 conn(0x7f78740b6170 legacy=0x7f78740b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:53.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f7872ffd700 1 -- 172.21.15.16:0/4128804921 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740b65e0 con 0x7f7874159810 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.425+0000 7f78737fe700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f78740b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7872ffd700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7874159810 0x7f78740ab040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7864009f20 tx=0x7f7864009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4986cac43aba3aac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7871ffb700 1 -- 172.21.15.16:0/4128804921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400e070 con 0x7f7874159810 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7871ffb700 1 -- 172.21.15.16:0/4128804921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400ba40 con 0x7f7874159810 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7871ffb700 1 -- 172.21.15.16:0/4128804921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640092e0 con 0x7f7874159810 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 -- 172.21.15.16:0/4128804921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7874159810 msgr2=0x7f78740ab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7874159810 0x7f78740ab040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7864009f20 tx=0x7f7864009840 comp rx=0 tx=0).stop 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 -- 172.21.15.16:0/4128804921 shutdown_connections 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f78740b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 --2- 172.21.15.16:0/4128804921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7874159810 0x7f78740ab040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 -- 172.21.15.16:0/4128804921 >> 172.21.15.16:0/4128804921 conn(0x7f787408a1e0 msgr2=0x7f7874099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:53.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 -- 172.21.15.16:0/4128804921 shutdown_connections 2023-10-06T11:41:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.426+0000 7f7879914700 1 -- 172.21.15.16:0/4128804921 wait complete. 2023-10-06T11:41:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 Processor -- start 2023-10-06T11:41:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 -- start start 2023-10-06T11:41:53.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f7874158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 0x7f78740c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c70e0 con 0x7f78740b6170 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78741593b0 con 0x7f7874159810 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f7879914700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740c6150 con 0x7f78740b3fc0 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f78727fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f78727fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57306/0 (socket says 172.21.15.16:57306) 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.428+0000 7f78727fc700 1 -- 172.21.15.16:0/1267493008 learned_addr learned my addr 172.21.15.16:0/1267493008 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f7872ffd700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f7874158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78737fe700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 0x7f78740c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 -- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 msgr2=0x7f7874158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f7874158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 -- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 msgr2=0x7f78740c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 0x7f78740c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 -- 172.21.15.16:0/1267493008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009ae0 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f7872ffd700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f7874158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78737fe700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 0x7f78740c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f78727fc700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b5e0 tx=0x7f785c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00f510 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c00f670 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740c6430 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00fa30 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.429+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78740c69f0 con 0x7f78740b6170 2023-10-06T11:41:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.430+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7874063e70 con 0x7f78740b6170 2023-10-06T11:41:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.430+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f785c01a400 con 0x7f78740b6170 2023-10-06T11:41:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.431+0000 7f786affd700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 0x7f78540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.431+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f785c083f10 con 0x7f78740b6170 2023-10-06T11:41:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.431+0000 7f7872ffd700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 0x7f78540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.432+0000 7f7872ffd700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 0x7f78540693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7864009880 tx=0x7f7864017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.435+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f785c0513f0 con 0x7f78740b6170 2023-10-06T11:41:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:53.630+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f787406d200 con 0x7f78740b6170 2023-10-06T11:41:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.483+0000 7f786affd700 1 -- 172.21.15.16:0/1267493008 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f785c054a10 con 0x7f78740b6170 2023-10-06T11:41:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 msgr2=0x7f78540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 0x7f78540693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7864009880 tx=0x7f7864017000 comp rx=0 tx=0).stop 2023-10-06T11:41:54.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 msgr2=0x7f78740c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b5e0 tx=0x7f785c009120 comp rx=0 tx=0).stop 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 shutdown_connections 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7854066f30 0x7f78540693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7874159810 0x7f78740c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f78740b3fc0 0x7f7874158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 --2- 172.21.15.16:0/1267493008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78740b6170 0x7f78740c5570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.488+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 >> 172.21.15.16:0/1267493008 conn(0x7f787408a1e0 msgr2=0x7f78740b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:54.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.489+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 shutdown_connections 2023-10-06T11:41:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.489+0000 7f7879914700 1 -- 172.21.15.16:0/1267493008 wait complete. 2023-10-06T11:41:54.497 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-10-06T11:41:54.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:41:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.638+0000 7f359080c700 1 Processor -- start 2023-10-06T11:41:54.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.638+0000 7f359080c700 1 -- start start 2023-10-06T11:41:54.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.639+0000 7f359080c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f358c1570d0 0x7f358c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:54.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.639+0000 7f359080c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c06a7f0 con 0x7f358c06cb20 2023-10-06T11:41:54.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.639+0000 7f359080c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c15a130 con 0x7f358c0b3000 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.639+0000 7f359080c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c15a410 con 0x7f358c1570d0 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f358c1570d0 0x7f358c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f358c06cb20 0x7f358c06aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46170/0 (socket says 172.21.15.16:46170) 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358ad9d700 1 -- 172.21.15.16:0/3254303412 learned_addr learned my addr 172.21.15.16:0/3254303412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f358c1570d0 0x7f358c0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57314/0 (socket says 172.21.15.16:57314) 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 -- 172.21.15.16:0/3254303412 >> v1:172.21.15.177:6789/0 conn(0x7f358c0b3000 legacy=0x7f358c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 -- 172.21.15.16:0/3254303412 >> v1:172.21.15.16:6789/0 conn(0x7f358c06cb20 legacy=0x7f358c06aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 -- 172.21.15.16:0/3254303412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c15a6f0 con 0x7f358c1570d0 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358959a700 1 -- 172.21.15.16:0/3254303412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890772863 0 0) 0x7f358c06a7f0 con 0x7f358c06cb20 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358a59c700 1 --2- 172.21.15.16:0/3254303412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c1570d0 0x7f358c0b2ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f357c0098e0 tx=0x7f357c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896cbe635ede4bc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:54.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358959a700 1 -- 172.21.15.16:0/3254303412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c009e80 con 0x7f358c1570d0 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358959a700 1 -- 172.21.15.16:0/3254303412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c00b820 con 0x7f358c1570d0 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.640+0000 7f358959a700 1 -- 172.21.15.16:0/3254303412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c00bba0 con 0x7f358c1570d0 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 -- 172.21.15.16:0/3254303412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c1570d0 msgr2=0x7f358c0b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 --2- 172.21.15.16:0/3254303412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c1570d0 0x7f358c0b2ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f357c0098e0 tx=0x7f357c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 -- 172.21.15.16:0/3254303412 shutdown_connections 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 --2- 172.21.15.16:0/3254303412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c1570d0 0x7f358c0b2ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 -- 172.21.15.16:0/3254303412 >> 172.21.15.16:0/3254303412 conn(0x7f358c068030 msgr2=0x7f358c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 -- 172.21.15.16:0/3254303412 shutdown_connections 2023-10-06T11:41:54.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.641+0000 7f359080c700 1 -- 172.21.15.16:0/3254303412 wait complete. 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.642+0000 7f359080c700 1 Processor -- start 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.642+0000 7f359080c700 1 -- start start 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f358c0b3000 0x7f358c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f358c1570d0 0x7f358c16d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c157a60 con 0x7f358c06cb20 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c157d40 con 0x7f358c1570d0 2023-10-06T11:41:54.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c158020 con 0x7f358c0b3000 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57320/0 (socket says 172.21.15.16:57320) 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 -- 172.21.15.16:0/4032630682 learned_addr learned my addr 172.21.15.16:0/4032630682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 -- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f358c0b3000 msgr2=0x7f358c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f358c0b3000 0x7f358c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 -- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f358c1570d0 msgr2=0x7f358c16d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f358c1570d0 0x7f358c16d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 -- 172.21.15.16:0/4032630682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c009420 con 0x7f358c06cb20 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f358a59c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f357c0030c0 tx=0x7f357c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c00ba10 con 0x7f358c06cb20 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c020070 con 0x7f358c06cb20 2023-10-06T11:41:54.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c158300 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c01b6a0 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.643+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0aabd0 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.644+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c0ae930 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.645+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f357c022030 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.645+0000 7f35827fc700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.645+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f357c086430 con 0x7f358c06cb20 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.646+0000 7f3589d9b700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:54.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.646+0000 7f3589d9b700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3574007fd0 tx=0x7f3574007b20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:54.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.650+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f358c0ae930 con 0x7f358c06cb20 2023-10-06T11:41:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:54.844+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f358c0ae930 con 0x7f358c06cb20 2023-10-06T11:41:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.486+0000 7f35827fc700 1 -- 172.21.15.16:0/4032630682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f358c0ae930 con 0x7f358c06cb20 2023-10-06T11:41:55.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.490+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 msgr2=0x7f356c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:55.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.490+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 0x7f356c0693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3574007fd0 tx=0x7f3574007b20 comp rx=0 tx=0).stop 2023-10-06T11:41:55.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.490+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 msgr2=0x7f358c0aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.490+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f357c0030c0 tx=0x7f357c016000 comp rx=0 tx=0).stop 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 shutdown_connections 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f358c06cb20 0x7f358c0aa690 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f357c0030c0 tx=0x7f357c016000 comp rx=0 tx=0).stop 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f356c066f30 0x7f356c0693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f358c1570d0 0x7f358c16d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 --2- 172.21.15.16:0/4032630682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f358c0b3000 0x7f358c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 >> 172.21.15.16:0/4032630682 conn(0x7f358c068030 msgr2=0x7f358c06c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:55.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.491+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 shutdown_connections 2023-10-06T11:41:55.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.492+0000 7f359080c700 1 -- 172.21.15.16:0/4032630682 wait complete. 2023-10-06T11:41:55.504 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-10-06T11:41:55.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:41:55.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.641+0000 7fa36b7b4700 1 Processor -- start 2023-10-06T11:41:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 -- start start 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 0x7fa3640b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa364159b10 0x7fa3640cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3640cead0 con 0x7fa3640cda10 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640affc0 con 0x7fa364159b10 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.642+0000 7fa36b7b4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640cdec0 con 0x7fa364159740 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa364159b10 0x7fa3640cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa364159b10 0x7fa3640cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57336/0 (socket says 172.21.15.16:57336) 2023-10-06T11:41:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 -- 172.21.15.16:0/2321312569 learned_addr learned my addr 172.21.15.16:0/2321312569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369550700 1 --2- 172.21.15.16:0/2321312569 >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 0x7fa3640b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 -- 172.21.15.16:0/2321312569 >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 msgr2=0x7fa3640b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 --2- 172.21.15.16:0/2321312569 >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 0x7fa3640b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 -- 172.21.15.16:0/2321312569 >> v1:172.21.15.177:6789/0 conn(0x7fa3640cda10 legacy=0x7fa3640b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 -- 172.21.15.16:0/2321312569 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640ce1a0 con 0x7fa364159b10 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369550700 1 --2- 172.21.15.16:0/2321312569 >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 0x7fa3640b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa369d51700 1 --2- 172.21.15.16:0/2321312569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159b10 0x7fa3640cef60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa364089030 tx=0x7fa35c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b8d13b647ea921 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa363fff700 1 -- 172.21.15.16:0/2321312569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00b820 con 0x7fa364159b10 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa363fff700 1 -- 172.21.15.16:0/2321312569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00b980 con 0x7fa364159b10 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.643+0000 7fa363fff700 1 -- 172.21.15.16:0/2321312569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00bd20 con 0x7fa364159b10 2023-10-06T11:41:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 -- 172.21.15.16:0/2321312569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159b10 msgr2=0x7fa3640cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/2321312569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159b10 0x7fa3640cef60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa364089030 tx=0x7fa35c009250 comp rx=0 tx=0).stop 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 -- 172.21.15.16:0/2321312569 shutdown_connections 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/2321312569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159b10 0x7fa3640cef60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/2321312569 >> v2:172.21.15.177:3300/0 conn(0x7fa364159740 0x7fa3640b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 -- 172.21.15.16:0/2321312569 >> 172.21.15.16:0/2321312569 conn(0x7fa364099a40 msgr2=0x7fa36406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 -- 172.21.15.16:0/2321312569 shutdown_connections 2023-10-06T11:41:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.644+0000 7fa36b7b4700 1 -- 172.21.15.16:0/2321312569 wait complete. 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.645+0000 7fa36b7b4700 1 Processor -- start 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.645+0000 7fa36b7b4700 1 -- start start 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3640cda10 0x7fa36409dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa364159b10 0x7fa36409b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415a680 con 0x7fa364159740 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415a070 con 0x7fa3640cda10 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa36b7b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36409e280 con 0x7fa364159b10 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa369550700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3640cda10 0x7fa36409dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57350/0 (socket says 172.21.15.16:57350) 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 -- 172.21.15.16:0/1053858210 learned_addr learned my addr 172.21.15.16:0/1053858210 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa369d51700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa364159b10 0x7fa36409b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 -- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa364159b10 msgr2=0x7fa36409b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa364159b10 0x7fa36409b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 -- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3640cda10 msgr2=0x7fa36409dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3640cda10 0x7fa36409dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 -- 172.21.15.16:0/1053858210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0094f0 con 0x7fa364159740 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.646+0000 7fa368d4f700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b5e0 tx=0x7fa358009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.647+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800f510 con 0x7fa364159740 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.647+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35800f670 con 0x7fa364159740 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.647+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36409a3b0 con 0x7fa364159740 2023-10-06T11:41:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.647+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800fa30 con 0x7fa364159740 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.647+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36415a310 con 0x7fa364159740 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.648+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36415a7f0 con 0x7fa364159740 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.648+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa35801a400 con 0x7fa364159740 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.649+0000 7fa3617fa700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.649+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fa358084020 con 0x7fa364159740 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.649+0000 7fa369550700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.649+0000 7fa369550700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa35400d6e0 tx=0x7fa354007850 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.653+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa358051210 con 0x7fa364159740 2023-10-06T11:41:55.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:55.890+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa36404c380 con 0x7fa364159740 2023-10-06T11:41:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.520+0000 7fa3617fa700 1 -- 172.21.15.16:0/1053858210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa358058e60 con 0x7fa364159740 2023-10-06T11:41:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 msgr2=0x7fa3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa35400d6e0 tx=0x7fa354007850 comp rx=0 tx=0).stop 2023-10-06T11:41:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 msgr2=0x7fa36415c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa35800b5e0 tx=0x7fa358009120 comp rx=0 tx=0).stop 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 shutdown_connections 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa364159b10 0x7fa36409b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.522+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa3640cda10 0x7fa36409dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.523+0000 7fa36b7b4700 1 --2- 172.21.15.16:0/1053858210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa364159740 0x7fa36415c730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.523+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 >> 172.21.15.16:0/1053858210 conn(0x7fa364099a40 msgr2=0x7fa3641564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.523+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 shutdown_connections 2023-10-06T11:41:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.523+0000 7fa36b7b4700 1 -- 172.21.15.16:0/1053858210 wait complete. 2023-10-06T11:41:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T11:41:56.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.663+0000 7fc98f296700 1 Processor -- start 2023-10-06T11:41:56.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.664+0000 7fc98f296700 1 -- start start 2023-10-06T11:41:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98f296700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc98815cc20 0x7fc9880cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98f296700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 0x7fc9880ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98f296700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880cdb10 con 0x7fc9880cf280 2023-10-06T11:41:56.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98f296700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880b3110 con 0x7fc98815cc20 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98f296700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880ccb20 con 0x7fc9880cd6a0 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc98815cc20 0x7fc9880cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc98815cc20 0x7fc9880cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57354/0 (socket says 172.21.15.16:57354) 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 -- 172.21.15.16:0/1293582290 learned_addr learned my addr 172.21.15.16:0/1293582290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98c831700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 0x7fc9880ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 -- 172.21.15.16:0/1293582290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 msgr2=0x7fc9880ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 0x7fc9880ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 -- 172.21.15.16:0/1293582290 >> v1:172.21.15.177:6789/0 conn(0x7fc9880cf280 legacy=0x7fc98815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 -- 172.21.15.16:0/1293582290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880cbf10 con 0x7fc98815cc20 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98c831700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 0x7fc9880ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc98d032700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc98815cc20 0x7fc9880cd160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9780099b0 tx=0x7fc978009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf7ce5c0e3e1d54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:56.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc987fff700 1 -- 172.21.15.16:0/1293582290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800b820 con 0x7fc98815cc20 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc987fff700 1 -- 172.21.15.16:0/1293582290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97800b980 con 0x7fc98815cc20 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.665+0000 7fc987fff700 1 -- 172.21.15.16:0/1293582290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97800bd40 con 0x7fc98815cc20 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 -- 172.21.15.16:0/1293582290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc98815cc20 msgr2=0x7fc9880cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc98815cc20 0x7fc9880cd160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc9780099b0 tx=0x7fc978009250 comp rx=0 tx=0).stop 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 -- 172.21.15.16:0/1293582290 shutdown_connections 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc98815cc20 0x7fc9880cd160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 --2- 172.21.15.16:0/1293582290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cd6a0 0x7fc9880ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 -- 172.21.15.16:0/1293582290 >> 172.21.15.16:0/1293582290 conn(0x7fc988099b40 msgr2=0x7fc98806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 -- 172.21.15.16:0/1293582290 shutdown_connections 2023-10-06T11:41:56.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.666+0000 7fc98f296700 1 -- 172.21.15.16:0/1293582290 wait complete. 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.667+0000 7fc98f296700 1 Processor -- start 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.667+0000 7fc98f296700 1 -- start start 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cf280 0x7fc988158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 0x7fc98809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc988158c60 con 0x7fc9880cd6a0 2023-10-06T11:41:56.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98809d3f0 con 0x7fc98815cc20 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98f296700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98809d6d0 con 0x7fc9880cf280 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57356/0 (socket says 172.21.15.16:57356) 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 -- 172.21.15.16:0/3623146533 learned_addr learned my addr 172.21.15.16:0/3623146533 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98c831700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cf280 0x7fc988158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d833700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 0x7fc98809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 -- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cf280 msgr2=0x7fc988158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cf280 0x7fc988158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 -- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 msgr2=0x7fc98809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 0x7fc98809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d032700 1 -- 172.21.15.16:0/3623146533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9780094f0 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.668+0000 7fc98d833700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 0x7fc98809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc98d032700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc988089390 tx=0x7fc978016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc978011bd0 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc978020070 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97801b450 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98809d9b0 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.669+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98809df70 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.670+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98809e160 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.670+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc978022030 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.670+0000 7fc9857fa700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.671+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc978086450 con 0x7fc9880cd6a0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.671+0000 7fc98c831700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:56.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.671+0000 7fc98c831700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 0x7fc9740693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b2870 tx=0x7fc97c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:56.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.675+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc978053560 con 0x7fc9880cd6a0 2023-10-06T11:41:56.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.869+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc98809e160 con 0x7fc9880cd6a0 2023-10-06T11:41:56.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.870+0000 7fc9857fa700 1 -- 172.21.15.16:0/3623146533 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fc978056b80 con 0x7fc9880cd6a0 2023-10-06T11:41:56.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:56.878 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:55.892782+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 msgr2=0x7fc9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 0x7fc9740693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b2870 tx=0x7fc97c008040 comp rx=0 tx=0).stop 2023-10-06T11:41:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 msgr2=0x7fc98816a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:56.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc988089390 tx=0x7fc978016000 comp rx=0 tx=0).stop 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 shutdown_connections 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9880cd6a0 0x7fc98816a6a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc974066f30 0x7fc9740693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc98815cc20 0x7fc98809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 --2- 172.21.15.16:0/3623146533 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9880cf280 0x7fc988158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 >> 172.21.15.16:0/3623146533 conn(0x7fc988099b40 msgr2=0x7fc988159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 shutdown_connections 2023-10-06T11:41:56.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:56.872+0000 7fc98f296700 1 -- 172.21.15.16:0/3623146533 wait complete. 2023-10-06T11:41:56.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:41:57.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.003+0000 7fbce3c8c700 1 Processor -- start 2023-10-06T11:41:57.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.004+0000 7fbce3c8c700 1 -- start start 2023-10-06T11:41:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.004+0000 7fbce3c8c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcdc15cc20 0x7fbcdc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.004+0000 7fbce3c8c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 0x7fbcdc0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.004+0000 7fbce3c8c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc0cdb10 con 0x7fbcdc0cf280 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce3c8c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc0b3110 con 0x7fbcdc15cc20 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce3c8c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0ccb20 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce2229700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbcdc0cf280 0x7fbcdc15d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46184/0 (socket says 172.21.15.16:46184) 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce2229700 1 -- 172.21.15.16:0/4076065819 learned_addr learned my addr 172.21.15.16:0/4076065819 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 --2- 172.21.15.16:0/4076065819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 0x7fbcdc0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1a28700 1 --2- 172.21.15.16:0/4076065819 >> v2:172.21.15.177:3300/0 conn(0x7fbcdc15cc20 0x7fbcdc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189009566 0 0) 0x7fbcdc0cdb10 con 0x7fbcdc0cf280 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcc4003570 con 0x7fbcdc0cf280 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 -- 172.21.15.16:0/4076065819 >> v2:172.21.15.177:3300/0 conn(0x7fbcdc15cc20 msgr2=0x7fbcdc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 --2- 172.21.15.16:0/4076065819 >> v2:172.21.15.177:3300/0 conn(0x7fbcdc15cc20 0x7fbcdc0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393620181 0 0) 0x7fbcc4003570 con 0x7fbcdc0cf280 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 -- 172.21.15.16:0/4076065819 >> v1:172.21.15.16:6789/0 conn(0x7fbcdc0cf280 legacy=0x7fbcdc15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 -- 172.21.15.16:0/4076065819 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0cbf10 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.005+0000 7fbce1227700 1 --2- 172.21.15.16:0/4076065819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 0x7fbcdc0ced40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098e0 tx=0x7fbcd00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce18c7294116f0be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4001470 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd0011420 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce0a26700 1 -- 172.21.15.16:0/4076065819 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd00117a0 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 -- 172.21.15.16:0/4076065819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 msgr2=0x7fbcdc0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/4076065819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 0x7fbcdc0ced40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbcd00098e0 tx=0x7fbcd00091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 -- 172.21.15.16:0/4076065819 shutdown_connections 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/4076065819 >> v2:172.21.15.177:3300/0 conn(0x7fbcdc15cc20 0x7fbcdc0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/4076065819 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc0cd6a0 0x7fbcdc0ced40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 -- 172.21.15.16:0/4076065819 >> 172.21.15.16:0/4076065819 conn(0x7fbcdc099b40 msgr2=0x7fbcdc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 -- 172.21.15.16:0/4076065819 shutdown_connections 2023-10-06T11:41:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.006+0000 7fbce3c8c700 1 -- 172.21.15.16:0/4076065819 wait complete. 2023-10-06T11:41:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.007+0000 7fbce3c8c700 1 Processor -- start 2023-10-06T11:41:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.007+0000 7fbce3c8c700 1 -- start start 2023-10-06T11:41:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcdc0cd6a0 0x7fbcdc1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 0x7fbcdc0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0c7d50 con 0x7fbcdc0cf280 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0b3a40 con 0x7fbcdc0cd6a0 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce3c8c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc0b3c70 con 0x7fbcdc15cc20 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57370/0 (socket says 172.21.15.16:57370) 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 -- 172.21.15.16:0/3025405921 learned_addr learned my addr 172.21.15.16:0/3025405921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce2229700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 0x7fbcdc0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 -- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 msgr2=0x7fbcdc0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 0x7fbcdc0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 -- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcdc0cd6a0 msgr2=0x7fbcdc1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcdc0cd6a0 0x7fbcdc1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce1227700 1 -- 172.21.15.16:0/3025405921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd0009420 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.008+0000 7fbce2229700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 0x7fbcdc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbce1227700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0ccc60 tx=0x7fbcd0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0000ce0 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd001cc50 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc0b3f50 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd001a450 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.009+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc151ea0 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.010+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc0b42f0 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.010+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcd001a840 con 0x7fbcdc0cf280 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.011+0000 7fbcd9ffb700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.011+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbcd0085e50 con 0x7fbcdc0cf280 2023-10-06T11:41:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.011+0000 7fbce1a28700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.011+0000 7fbce1a28700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 0x7fbcc80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e260 tx=0x7fbccc00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.015+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcd0053010 con 0x7fbcdc0cf280 2023-10-06T11:41:57.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.210+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcdc152090 con 0x7fbcdc0cf280 2023-10-06T11:41:57.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.211+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/3025405921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fbcd0029020 con 0x7fbcdc0cf280 2023-10-06T11:41:57.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:57.214 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:55.892782+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:41:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 msgr2=0x7fbcc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 0x7fbcc80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e260 tx=0x7fbccc00b040 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 msgr2=0x7fbcdc0c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc0ccc60 tx=0x7fbcd0016000 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 shutdown_connections 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbcc8066f30 0x7fbcc80693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcdc15cc20 0x7fbcdc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcdc0cd6a0 0x7fbcdc1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 --2- 172.21.15.16:0/3025405921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc0cf280 0x7fbcdc0c7810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 >> 172.21.15.16:0/3025405921 conn(0x7fbcdc099b40 msgr2=0x7fbcdc0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:57.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 shutdown_connections 2023-10-06T11:41:57.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.213+0000 7fbce3c8c700 1 -- 172.21.15.16:0/3025405921 wait complete. 2023-10-06T11:41:57.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:41:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.349+0000 7faeb1505700 1 Processor -- start 2023-10-06T11:41:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.349+0000 7faeb1505700 1 -- start start 2023-10-06T11:41:57.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeb1505700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faeac159740 0x7faeac0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeb1505700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac0cda10 0x7faeac0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeb1505700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac0cead0 con 0x7faeac159b10 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeb1505700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac0affc0 con 0x7faeac159740 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeb1505700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cdec0 con 0x7faeac0cda10 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeaaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faeac159740 0x7faeac0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeaaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faeac159740 0x7faeac0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57374/0 (socket says 172.21.15.16:57374) 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.350+0000 7faeaaffd700 1 -- 172.21.15.16:0/2906059977 learned_addr learned my addr 172.21.15.16:0/2906059977 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaa7fc700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac0cda10 0x7faeac0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaaffd700 1 -- 172.21.15.16:0/2906059977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac0cda10 msgr2=0x7faeac0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaaffd700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac0cda10 0x7faeac0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaaffd700 1 -- 172.21.15.16:0/2906059977 >> v1:172.21.15.177:6789/0 conn(0x7faeac159b10 legacy=0x7faeac15d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaaffd700 1 -- 172.21.15.16:0/2906059977 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0ce1a0 con 0x7faeac159740 2023-10-06T11:41:57.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeaaffd700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009f20 tx=0x7fae9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e792343ec18a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faea9ffb700 1 -- 172.21.15.16:0/2906059977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00e070 con 0x7faeac159740 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faea9ffb700 1 -- 172.21.15.16:0/2906059977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00ba80 con 0x7faeac159740 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faea9ffb700 1 -- 172.21.15.16:0/2906059977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0092e0 con 0x7faeac159740 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.351+0000 7faeb1505700 1 -- 172.21.15.16:0/2906059977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 msgr2=0x7faeac0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009f20 tx=0x7fae9c009840 comp rx=0 tx=0).stop 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 -- 172.21.15.16:0/2906059977 shutdown_connections 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 --2- 172.21.15.16:0/2906059977 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac0cda10 0x7faeac0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 -- 172.21.15.16:0/2906059977 >> 172.21.15.16:0/2906059977 conn(0x7faeac099a40 msgr2=0x7faeac06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 -- 172.21.15.16:0/2906059977 shutdown_connections 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.352+0000 7faeb1505700 1 -- 172.21.15.16:0/2906059977 wait complete. 2023-10-06T11:41:57.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.353+0000 7faeb1505700 1 Processor -- start 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.353+0000 7faeb1505700 1 -- start start 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeac0cda10 0x7faeac1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 0x7faeac15f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0ae9b0 con 0x7faeac159740 2023-10-06T11:41:57.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0af210 con 0x7faeac0cda10 2023-10-06T11:41:57.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0b49c0 con 0x7faeac159b10 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57380/0 (socket says 172.21.15.16:57380) 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 -- 172.21.15.16:0/2085188084 learned_addr learned my addr 172.21.15.16:0/2085188084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaaffd700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeac0cda10 0x7faeac1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeab7fe700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 0x7faeac15f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 -- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 msgr2=0x7faeac15f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 0x7faeac15f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 -- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeac0cda10 msgr2=0x7faeac1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeac0cda10 0x7faeac1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 -- 172.21.15.16:0/2085188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009ae0 con 0x7faeac159740 2023-10-06T11:41:57.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeab7fe700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 0x7faeac15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeaa7fc700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d890 tx=0x7fae94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940125f0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae94012750 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94012ad0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac0aebc0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.354+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac0ac1c0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.355+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac0af3a0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.356+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae9401e460 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.356+0000 7faea2ffd700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 0x7fae8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.356+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae94084cb0 con 0x7faeac159740 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.356+0000 7faeaaffd700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 0x7fae8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.357+0000 7faeaaffd700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 0x7fae8c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faeac15eb10 tx=0x7fae9c018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:57.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.360+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae94051dc0 con 0x7faeac159740 2023-10-06T11:41:57.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:57.600+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faeac159210 con 0x7faeac159740 2023-10-06T11:41:58.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.529+0000 7faea2ffd700 1 -- 172.21.15.16:0/2085188084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae940553e0 con 0x7faeac159740 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 msgr2=0x7fae8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 0x7fae8c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faeac15eb10 tx=0x7fae9c018000 comp rx=0 tx=0).stop 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 msgr2=0x7faeac0b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fae9400d890 tx=0x7fae94008040 comp rx=0 tx=0).stop 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 shutdown_connections 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae8c066f30 0x7fae8c0693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faeac159b10 0x7faeac15f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faeac0cda10 0x7faeac1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 --2- 172.21.15.16:0/2085188084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeac159740 0x7faeac0b3f90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 >> 172.21.15.16:0/2085188084 conn(0x7faeac099a40 msgr2=0x7faeac0b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 shutdown_connections 2023-10-06T11:41:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.532+0000 7faeb1505700 1 -- 172.21.15.16:0/2085188084 wait complete. 2023-10-06T11:41:58.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:41:58.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:58.612 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:41:58.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.680+0000 7fc0e8ac7700 1 Processor -- start 2023-10-06T11:41:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.680+0000 7fc0e8ac7700 1 -- start start 2023-10-06T11:41:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e8ac7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 0x7fc0e40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e8ac7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e40cead0 con 0x7fc0e4159b10 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e8ac7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e40affc0 con 0x7fc0e4159740 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e8ac7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e40cdec0 con 0x7fc0e40cda10 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e2d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc0e4159b10 0x7fc0e40cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56886/0 (socket says 172.21.15.16:56886) 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e2d9d700 1 -- 172.21.15.16:0/2804534311 learned_addr learned my addr 172.21.15.16:0/2804534311 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e1d9b700 1 --2- 172.21.15.16:0/2804534311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 0x7fc0e40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436871503 0 0) 0x7fc0e40cead0 con 0x7fc0e4159b10 2023-10-06T11:41:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0cc003570 con 0x7fc0e4159b10 2023-10-06T11:41:58.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056959716 0 0) 0x7fc0e40affc0 con 0x7fc0e4159740 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0e40cead0 con 0x7fc0e4159740 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e1d9b700 1 -- 172.21.15.16:0/2804534311 >> v1:172.21.15.177:6789/0 conn(0x7fc0e4159740 legacy=0x7fc0e40b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e1d9b700 1 -- 172.21.15.16:0/2804534311 >> v1:172.21.15.16:6789/0 conn(0x7fc0e4159b10 legacy=0x7fc0e40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e1d9b700 1 -- 172.21.15.16:0/2804534311 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40ce1a0 con 0x7fc0e40cda10 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.681+0000 7fc0e1d9b700 1 --2- 172.21.15.16:0/2804534311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 0x7fc0e40b40a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009910 tx=0x7fc0d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a2ca56ae98ef506 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8011630 con 0x7fc0e40cda10 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d8011790 con 0x7fc0e40cda10 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e159a700 1 -- 172.21.15.16:0/2804534311 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8011b30 con 0x7fc0e40cda10 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/2804534311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 msgr2=0x7fc0e40b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/2804534311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 0x7fc0e40b40a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009910 tx=0x7fc0d80091a0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/2804534311 shutdown_connections 2023-10-06T11:41:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/2804534311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e40cda10 0x7fc0e40b40a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/2804534311 >> 172.21.15.16:0/2804534311 conn(0x7fc0e4099a40 msgr2=0x7fc0e406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/2804534311 shutdown_connections 2023-10-06T11:41:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.682+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/2804534311 wait complete. 2023-10-06T11:41:58.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.683+0000 7fc0e8ac7700 1 Processor -- start 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.683+0000 7fc0e8ac7700 1 -- start start 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e4159740 0x7fc0e4151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 0x7fc0e4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e415afc0 con 0x7fc0e40cda10 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e414d1b0 con 0x7fc0e4159b10 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e8ac7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e415b910 con 0x7fc0e4159740 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42920/0 (socket says 172.21.15.16:42920) 2023-10-06T11:41:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e259c700 1 -- 172.21.15.16:0/3873765222 learned_addr learned my addr 172.21.15.16:0/3873765222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e1d9b700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e4159740 0x7fc0e4151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.684+0000 7fc0e2d9d700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 0x7fc0e4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 -- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e4159740 msgr2=0x7fc0e4151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e4159740 0x7fc0e4151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 -- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 msgr2=0x7fc0e4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 0x7fc0e4159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 -- 172.21.15.16:0/3873765222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8009420 con 0x7fc0e40cda10 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e2d9d700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 0x7fc0e4159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e259c700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c790 tx=0x7fc0d4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010510 con 0x7fc0e40cda10 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d4010670 con 0x7fc0e40cda10 2023-10-06T11:41:58.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4010a30 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e4151c40 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.685+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e415ace0 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.686+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e404c380 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.686+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0d401b3b0 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.687+0000 7fc0d27fc700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 0x7fc0c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.687+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc0d4083f80 con 0x7fc0e40cda10 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.687+0000 7fc0e1d9b700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 0x7fc0c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:58.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.687+0000 7fc0e1d9b700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 0x7fc0c40693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009860 tx=0x7fc0d8017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:58.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.691+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0d40b55a0 con 0x7fc0e40cda10 2023-10-06T11:41:58.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:58.930+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc0e40d0470 con 0x7fc0e40cda10 2023-10-06T11:41:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.530+0000 7fc0d27fc700 1 -- 172.21.15.16:0/3873765222 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc0d4017020 con 0x7fc0e40cda10 2023-10-06T11:41:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 msgr2=0x7fc0c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 0x7fc0c40693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8009860 tx=0x7fc0d8017000 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 msgr2=0x7fc0e414cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400c790 tx=0x7fc0d4007e10 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 shutdown_connections 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0e40cda10 0x7fc0e414cc70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc0c4066f30 0x7fc0c40693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0e4159b10 0x7fc0e4159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 --2- 172.21.15.16:0/3873765222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0e4159740 0x7fc0e4151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 >> 172.21.15.16:0/3873765222 conn(0x7fc0e4099a40 msgr2=0x7fc0e4156930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 shutdown_connections 2023-10-06T11:41:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.532+0000 7fc0e8ac7700 1 -- 172.21.15.16:0/3873765222 wait complete. 2023-10-06T11:41:59.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:41:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.713+0000 7f55f9dbd700 1 Processor -- start 2023-10-06T11:41:59.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.713+0000 7f55f9dbd700 1 -- start start 2023-10-06T11:41:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f9dbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f415d920 0x7f55f40b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:59.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f9dbd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f415a650 con 0x7f55f4159790 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f9dbd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40b61c0 con 0x7f55f41593c0 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f9dbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f4159d20 con 0x7f55f415d920 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f415d920 0x7f55f40b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f95bc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55f4159790 0x7f55f40b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56888/0 (socket says 172.21.15.16:56888) 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f95bc700 1 -- 172.21.15.16:0/1397491200 learned_addr learned my addr 172.21.15.16:0/1397491200 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f415d920 0x7f55f40b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42930/0 (socket says 172.21.15.16:42930) 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f3fff700 1 -- 172.21.15.16:0/1397491200 >> v1:172.21.15.177:6789/0 conn(0x7f55f41593c0 legacy=0x7f55f415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f3fff700 1 -- 172.21.15.16:0/1397491200 >> v1:172.21.15.16:6789/0 conn(0x7f55f4159790 legacy=0x7f55f40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f3fff700 1 -- 172.21.15.16:0/1397491200 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f415a000 con 0x7f55f415d920 2023-10-06T11:41:59.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.714+0000 7f55f37fe700 1 -- 172.21.15.16:0/1397491200 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356912419 0 0) 0x7f55f415a650 con 0x7f55f4159790 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f3fff700 1 --2- 172.21.15.16:0/1397491200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f415d920 0x7f55f40b53f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009eb0 tx=0x7f55e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809bf294fa386be3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f37fe700 1 -- 172.21.15.16:0/1397491200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e400e070 con 0x7f55f415d920 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f37fe700 1 -- 172.21.15.16:0/1397491200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e400bc30 con 0x7f55f415d920 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f37fe700 1 -- 172.21.15.16:0/1397491200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4012430 con 0x7f55f415d920 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 -- 172.21.15.16:0/1397491200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f415d920 msgr2=0x7f55f40b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/1397491200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f415d920 0x7f55f40b53f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009eb0 tx=0x7f55e4009780 comp rx=0 tx=0).stop 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 -- 172.21.15.16:0/1397491200 shutdown_connections 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/1397491200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f415d920 0x7f55f40b53f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 -- 172.21.15.16:0/1397491200 >> 172.21.15.16:0/1397491200 conn(0x7f55f408a020 msgr2=0x7f55f4099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 -- 172.21.15.16:0/1397491200 shutdown_connections 2023-10-06T11:41:59.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.715+0000 7f55f9dbd700 1 -- 172.21.15.16:0/1397491200 wait complete. 2023-10-06T11:41:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.716+0000 7f55f9dbd700 1 Processor -- start 2023-10-06T11:41:59.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 -- start start 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 0x7f55f41698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f415d920 0x7f55f415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f41522c0 con 0x7f55f4159790 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f416aad0 con 0x7f55f415d920 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.717+0000 7f55f9dbd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4169f10 con 0x7f55f41593c0 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42944/0 (socket says 172.21.15.16:42944) 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 -- 172.21.15.16:0/4047298868 learned_addr learned my addr 172.21.15.16:0/4047298868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f95bc700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f415d920 0x7f55f415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:59.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f8dbb700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 0x7f55f41698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 -- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 msgr2=0x7f55f41698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 0x7f55f41698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 -- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f415d920 msgr2=0x7f55f415bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f415d920 0x7f55f415bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 -- 172.21.15.16:0/4047298868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e4009a20 con 0x7f55f4159790 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f8dbb700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 0x7f55f41698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f3fff700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55e4013000 tx=0x7f55e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4012e60 con 0x7f55f4159790 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e401b630 con 0x7f55f4159790 2023-10-06T11:41:59.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f416a1f0 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e401b9d0 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.718+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f416a7b0 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.719+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f416acf0 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.720+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55e401bd40 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.720+0000 7f55f0ff9700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.720+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55e40863e0 con 0x7f55f4159790 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.720+0000 7f55f8dbb700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:41:59.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.721+0000 7f55f8dbb700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55f409e410 tx=0x7f55dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:41:59.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.724+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55f416acf0 con 0x7f55f4159790 2023-10-06T11:41:59.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.963+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f416acf0 con 0x7f55f4159790 2023-10-06T11:41:59.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.964+0000 7f55f0ff9700 1 -- 172.21.15.16:0/4047298868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f55f416acf0 con 0x7f55f4159790 2023-10-06T11:41:59.966 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:41:59.966 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:41:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 msgr2=0x7f55d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f55f409e410 tx=0x7f55dc009040 comp rx=0 tx=0).stop 2023-10-06T11:41:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 msgr2=0x7f55f4151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:41:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f55e4013000 tx=0x7f55e4017000 comp rx=0 tx=0).stop 2023-10-06T11:41:59.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 shutdown_connections 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f55f415d920 0x7f55f415bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f55f41593c0 0x7f55f41698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 --2- 172.21.15.16:0/4047298868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f4159790 0x7f55f4151d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 >> 172.21.15.16:0/4047298868 conn(0x7f55f408a020 msgr2=0x7f55f415ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 shutdown_connections 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:41:59.966+0000 7f55f9dbd700 1 -- 172.21.15.16:0/4047298868 wait complete. 2023-10-06T11:41:59.969 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:41:59.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.098+0000 7f37fcdab700 1 Processor -- start 2023-10-06T11:42:00.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.098+0000 7f37fcdab700 1 -- start start 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37fcdab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37fcdab700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f37f806c410 0x7f37f806c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37fcdab700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f815da60 con 0x7f37f80cc890 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37fcdab700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f815e530 con 0x7f37f806c410 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37fcdab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8067cd0 con 0x7f37f80af8d0 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f6d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f37f806c410 0x7f37f806c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f5d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f659c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f37f80cc890 0x7f37f80675c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48644/0 (socket says 172.21.15.16:48644) 2023-10-06T11:42:00.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f659c700 1 -- 172.21.15.16:0/1280364778 learned_addr learned my addr 172.21.15.16:0/1280364778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:00.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f559a700 1 -- 172.21.15.16:0/1280364778 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005595220 0 0) 0x7f37f815da60 con 0x7f37f80cc890 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f559a700 1 -- 172.21.15.16:0/1280364778 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e4003570 con 0x7f37f80cc890 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f6d9d700 1 -- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 msgr2=0x7f37f815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f6d9d700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f6d9d700 1 -- 172.21.15.16:0/1280364778 >> v1:172.21.15.177:6789/0 conn(0x7f37f80cc890 legacy=0x7f37f80675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f6d9d700 1 -- 172.21.15.16:0/1280364778 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8159dd0 con 0x7f37f806c410 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.099+0000 7f37f5d9b700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37f6d9d700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f806c890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009910 tx=0x7f37f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ca5f087b20f8b8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37f559a700 1 -- 172.21.15.16:0/1280364778 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011630 con 0x7f37f806c410 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37f559a700 1 -- 172.21.15.16:0/1280364778 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0011790 con 0x7f37f806c410 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37f559a700 1 -- 172.21.15.16:0/1280364778 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011b10 con 0x7f37f806c410 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 -- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 msgr2=0x7f37f806c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f806c890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009910 tx=0x7f37f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 -- 172.21.15.16:0/1280364778 shutdown_connections 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f806c890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 --2- 172.21.15.16:0/1280364778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 -- 172.21.15.16:0/1280364778 >> 172.21.15.16:0/1280364778 conn(0x7f37f806dfa0 msgr2=0x7f37f80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 -- 172.21.15.16:0/1280364778 shutdown_connections 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.100+0000 7f37fcdab700 1 -- 172.21.15.16:0/1280364778 wait complete. 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.101+0000 7f37fcdab700 1 Processor -- start 2023-10-06T11:42:00.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.101+0000 7f37fcdab700 1 -- start start 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f80ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f8154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f816a560 con 0x7f37f80cc890 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f8155970 con 0x7f37f806c410 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37fcdab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f815aa30 con 0x7f37f80af8d0 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37f6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37f6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42950/0 (socket says 172.21.15.16:42950) 2023-10-06T11:42:00.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37f6d9d700 1 -- 172.21.15.16:0/2831725492 learned_addr learned my addr 172.21.15.16:0/2831725492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37f5d9b700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f8154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.102+0000 7f37f659c700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f80ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 -- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 msgr2=0x7f37f8154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f8154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 -- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 msgr2=0x7f37f80ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f80ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 -- 172.21.15.16:0/2831725492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0009420 con 0x7f37f80cc890 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f659c700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f80ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37f6d9d700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37f80710c0 tx=0x7f37f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011b10 con 0x7f37f80cc890 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f001a430 con 0x7f37f80cc890 2023-10-06T11:42:00.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f0009420 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f001a7d0 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.103+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f815a1b0 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.104+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f804c380 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.104+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37f001ac00 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.105+0000 7f37ee7fc700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 0x7f37d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.105+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f37f00863f0 con 0x7f37f80cc890 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.105+0000 7f37f659c700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 0x7f37d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.105+0000 7f37f659c700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 0x7f37d80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005c50 tx=0x7f37e8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.109+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37f0053500 con 0x7f37f80cc890 2023-10-06T11:42:00.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.307+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37f815a780 con 0x7f37f80cc890 2023-10-06T11:42:00.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.308+0000 7f37ee7fc700 1 -- 172.21.15.16:0/2831725492 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f37f0056b20 con 0x7f37f80cc890 2023-10-06T11:42:00.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:00.310 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:00.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 msgr2=0x7f37d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 0x7f37d80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005c50 tx=0x7f37e8009040 comp rx=0 tx=0).stop 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 msgr2=0x7f37f8155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f37f80710c0 tx=0x7f37f0017000 comp rx=0 tx=0).stop 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 shutdown_connections 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f80cc890 0x7f37f8155430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d8066f30 0x7f37d80693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f806c410 0x7f37f80ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 --2- 172.21.15.16:0/2831725492 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f80af8d0 0x7f37f8154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 >> 172.21.15.16:0/2831725492 conn(0x7f37f806dfa0 msgr2=0x7f37f815dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 shutdown_connections 2023-10-06T11:42:00.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.310+0000 7f37fcdab700 1 -- 172.21.15.16:0/2831725492 wait complete. 2023-10-06T11:42:00.325 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:42:00.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.441+0000 7f2c350f2700 1 Processor -- start 2023-10-06T11:42:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 -- start start 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c300cd870 0x7f2c300ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c300cf280 0x7f2c300c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300ccb20 con 0x7f2c3015cc20 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300b3110 con 0x7f2c300cd870 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c350f2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300cbf10 con 0x7f2c300cf280 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c2e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c300cd870 0x7f2c300ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c2e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c300cd870 0x7f2c300ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42958/0 (socket says 172.21.15.16:42958) 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.442+0000 7f2c2e59c700 1 -- 172.21.15.16:0/291722877 learned_addr learned my addr 172.21.15.16:0/291722877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2f59e700 1 --2- 172.21.15.16:0/291722877 >> v2:172.21.15.177:3300/0 conn(0x7f2c300cf280 0x7f2c300c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2e59c700 1 -- 172.21.15.16:0/291722877 >> v2:172.21.15.177:3300/0 conn(0x7f2c300cf280 msgr2=0x7f2c300c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2e59c700 1 --2- 172.21.15.16:0/291722877 >> v2:172.21.15.177:3300/0 conn(0x7f2c300cf280 0x7f2c300c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2e59c700 1 -- 172.21.15.16:0/291722877 >> v1:172.21.15.177:6789/0 conn(0x7f2c3015cc20 legacy=0x7f2c300cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2e59c700 1 -- 172.21.15.16:0/291722877 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300cc1f0 con 0x7f2c300cd870 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2e59c700 1 --2- 172.21.15.16:0/291722877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c300ced40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009a270 tx=0x7f2c18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8c4bd7a960461c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2dd9b700 1 -- 172.21.15.16:0/291722877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800b820 con 0x7f2c300cd870 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2dd9b700 1 -- 172.21.15.16:0/291722877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1800b980 con 0x7f2c300cd870 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c2dd9b700 1 -- 172.21.15.16:0/291722877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800bd40 con 0x7f2c300cd870 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c350f2700 1 -- 172.21.15.16:0/291722877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 msgr2=0x7f2c300ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.443+0000 7f2c350f2700 1 --2- 172.21.15.16:0/291722877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c300ced40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009a270 tx=0x7f2c18009250 comp rx=0 tx=0).stop 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 -- 172.21.15.16:0/291722877 shutdown_connections 2023-10-06T11:42:00.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 --2- 172.21.15.16:0/291722877 >> v2:172.21.15.177:3300/0 conn(0x7f2c300cf280 0x7f2c300c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 --2- 172.21.15.16:0/291722877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c300ced40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 -- 172.21.15.16:0/291722877 >> 172.21.15.16:0/291722877 conn(0x7f2c30099b40 msgr2=0x7f2c3006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 -- 172.21.15.16:0/291722877 shutdown_connections 2023-10-06T11:42:00.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.444+0000 7f2c350f2700 1 -- 172.21.15.16:0/291722877 wait complete. 2023-10-06T11:42:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.445+0000 7f2c350f2700 1 Processor -- start 2023-10-06T11:42:00.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.445+0000 7f2c350f2700 1 -- start start 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300cf280 0x7f2c3014f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3015cc20 0x7f2c300ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3009b4e0 con 0x7f2c300cd870 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300ca760 con 0x7f2c3015cc20 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3009a8d0 con 0x7f2c300cf280 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42962/0 (socket says 172.21.15.16:42962) 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 -- 172.21.15.16:0/3675722631 learned_addr learned my addr 172.21.15.16:0/3675722631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2e59c700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300cf280 0x7f2c3014f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2f59e700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3015cc20 0x7f2c300ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 -- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300cf280 msgr2=0x7f2c3014f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300cf280 0x7f2c3014f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 -- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3015cc20 msgr2=0x7f2c300ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3015cc20 0x7f2c300ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 -- 172.21.15.16:0/3675722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180094f0 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c2ed9d700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c5d0 tx=0x7f2c20007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011590 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011710 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3009abb0 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011ad0 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.446+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3009b170 con 0x7f2c300cd870 2023-10-06T11:42:00.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.447+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3004c380 con 0x7f2c300cd870 2023-10-06T11:42:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.448+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c2001c460 con 0x7f2c300cd870 2023-10-06T11:42:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.448+0000 7f2c26ffd700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.448+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2c200842f0 con 0x7f2c300cd870 2023-10-06T11:42:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.448+0000 7f2c2e59c700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.449+0000 7f2c2e59c700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 0x7f2c100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009a270 tx=0x7f2c18016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.452+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c20051400 con 0x7f2c300cd870 2023-10-06T11:42:00.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.642+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c3009b620 con 0x7f2c10066f30 2023-10-06T11:42:00.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.651+0000 7f2c26ffd700 1 -- 172.21.15.16:0/3675722631 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286970 (secure 0 0 0) 0x7f2c1800baf0 con 0x7f2c10066f30 2023-10-06T11:42:00.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 msgr2=0x7f2c100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 0x7f2c100693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2c3009a270 tx=0x7f2c18016000 comp rx=0 tx=0).stop 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 msgr2=0x7f2c3009a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000c5d0 tx=0x7f2c20007f60 comp rx=0 tx=0).stop 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 shutdown_connections 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c300cd870 0x7f2c3009a2a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2c10066f30 0x7f2c100693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c3015cc20 0x7f2c300ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 --2- 172.21.15.16:0/3675722631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300cf280 0x7f2c3014f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 >> 172.21.15.16:0/3675722631 conn(0x7f2c30099b40 msgr2=0x7f2c3015b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 shutdown_connections 2023-10-06T11:42:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.654+0000 7f2c350f2700 1 -- 172.21.15.16:0/3675722631 wait complete. 2023-10-06T11:42:00.660 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:00.668 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-06T11:42:00.060721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381188,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593532,"statfs":{"total":773094113280,"available":770655776768,"internally_reserved":0,"allocated":2953216,"data_stored":1802764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.577239"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517366+0000","last_change":"2023-10-06T11:41:56.510681+0000","last_active":"2023-10-06T11:41:56.517366+0000","last_peered":"2023-10-06T11:41:56.517366+0000","last_clean":"2023-10-06T11:41:56.517366+0000","last_became_active":"2023-10-06T11:41:56.510274+0000","last_became_peered":"2023-10-06T11:41:56.510274+0000","last_unstale":"2023-10-06T11:41:56.517366+0000","last_undegraded":"2023-10-06T11:41:56.517366+0000","last_fullsized":"2023-10-06T11:41:56.517366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:25.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519077+0000","last_change":"2023-10-06T11:41:56.512482+0000","last_active":"2023-10-06T11:41:56.519077+0000","last_peered":"2023-10-06T11:41:56.519077+0000","last_clean":"2023-10-06T11:41:56.519077+0000","last_became_active":"2023-10-06T11:41:56.512222+0000","last_became_peered":"2023-10-06T11:41:56.512222+0000","last_unstale":"2023-10-06T11:41:56.519077+0000","last_undegraded":"2023-10-06T11:41:56.519077+0000","last_fullsized":"2023-10-06T11:41:56.519077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513731+0000","last_change":"2023-10-06T11:41:56.509931+0000","last_active":"2023-10-06T11:41:56.513731+0000","last_peered":"2023-10-06T11:41:56.513731+0000","last_clean":"2023-10-06T11:41:56.513731+0000","last_became_active":"2023-10-06T11:41:56.509393+0000","last_became_peered":"2023-10-06T11:41:56.509393+0000","last_unstale":"2023-10-06T11:41:56.513731+0000","last_undegraded":"2023-10-06T11:41:56.513731+0000","last_fullsized":"2023-10-06T11:41:56.513731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:56.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516425+0000","last_change":"2023-10-06T11:41:56.511208+0000","last_active":"2023-10-06T11:41:56.516425+0000","last_peered":"2023-10-06T11:41:56.516425+0000","last_clean":"2023-10-06T11:41:56.516425+0000","last_became_active":"2023-10-06T11:41:56.510934+0000","last_became_peered":"2023-10-06T11:41:56.510934+0000","last_unstale":"2023-10-06T11:41:56.516425+0000","last_undegraded":"2023-10-06T11:41:56.516425+0000","last_fullsized":"2023-10-06T11:41:56.516425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:21.656559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519759+0000","last_change":"2023-10-06T11:41:56.516474+0000","last_active":"2023-10-06T11:41:56.519759+0000","last_peered":"2023-10-06T11:41:56.519759+0000","last_clean":"2023-10-06T11:41:56.519759+0000","last_became_active":"2023-10-06T11:41:56.516168+0000","last_became_peered":"2023-10-06T11:41:56.516168+0000","last_unstale":"2023-10-06T11:41:56.519759+0000","last_undegraded":"2023-10-06T11:41:56.519759+0000","last_fullsized":"2023-10-06T11:41:56.519759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513822+0000","last_change":"2023-10-06T11:41:56.509889+0000","last_active":"2023-10-06T11:41:56.513822+0000","last_peered":"2023-10-06T11:41:56.513822+0000","last_clean":"2023-10-06T11:41:56.513822+0000","last_became_active":"2023-10-06T11:41:56.509181+0000","last_became_peered":"2023-10-06T11:41:56.509181+0000","last_unstale":"2023-10-06T11:41:56.513822+0000","last_undegraded":"2023-10-06T11:41:56.513822+0000","last_fullsized":"2023-10-06T11:41:56.513822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:42.296263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522246+0000","last_change":"2023-10-06T11:41:56.518850+0000","last_active":"2023-10-06T11:41:56.522246+0000","last_peered":"2023-10-06T11:41:56.522246+0000","last_clean":"2023-10-06T11:41:56.522246+0000","last_became_active":"2023-10-06T11:41:56.518578+0000","last_became_peered":"2023-10-06T11:41:56.518578+0000","last_unstale":"2023-10-06T11:41:56.522246+0000","last_undegraded":"2023-10-06T11:41:56.522246+0000","last_fullsized":"2023-10-06T11:41:56.522246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514003+0000","last_change":"2023-10-06T11:41:56.508470+0000","last_active":"2023-10-06T11:41:56.514003+0000","last_peered":"2023-10-06T11:41:56.514003+0000","last_clean":"2023-10-06T11:41:56.514003+0000","last_became_active":"2023-10-06T11:41:56.508167+0000","last_became_peered":"2023-10-06T11:41:56.508167+0000","last_unstale":"2023-10-06T11:41:56.514003+0000","last_undegraded":"2023-10-06T11:41:56.514003+0000","last_fullsized":"2023-10-06T11:41:56.514003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519630+0000","last_change":"2023-10-06T11:41:56.517434+0000","last_active":"2023-10-06T11:41:56.519630+0000","last_peered":"2023-10-06T11:41:56.519630+0000","last_clean":"2023-10-06T11:41:56.519630+0000","last_became_active":"2023-10-06T11:41:56.517143+0000","last_became_peered":"2023-10-06T11:41:56.517143+0000","last_unstale":"2023-10-06T11:41:56.519630+0000","last_undegraded":"2023-10-06T11:41:56.519630+0000","last_fullsized":"2023-10-06T11:41:56.519630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514049+0000","last_change":"2023-10-06T11:41:56.510953+0000","last_active":"2023-10-06T11:41:56.514049+0000","last_peered":"2023-10-06T11:41:56.514049+0000","last_clean":"2023-10-06T11:41:56.514049+0000","last_became_active":"2023-10-06T11:41:56.510684+0000","last_became_peered":"2023-10-06T11:41:56.510684+0000","last_unstale":"2023-10-06T11:41:56.514049+0000","last_undegraded":"2023-10-06T11:41:56.514049+0000","last_fullsized":"2023-10-06T11:41:56.514049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516780+0000","last_change":"2023-10-06T11:41:56.511114+0000","last_active":"2023-10-06T11:41:56.516780+0000","last_peered":"2023-10-06T11:41:56.516780+0000","last_clean":"2023-10-06T11:41:56.516780+0000","last_became_active":"2023-10-06T11:41:56.510653+0000","last_became_peered":"2023-10-06T11:41:56.510653+0000","last_unstale":"2023-10-06T11:41:56.516780+0000","last_undegraded":"2023-10-06T11:41:56.516780+0000","last_fullsized":"2023-10-06T11:41:56.516780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518118+0000","last_change":"2023-10-06T11:41:56.512980+0000","last_active":"2023-10-06T11:41:56.518118+0000","last_peered":"2023-10-06T11:41:56.518118+0000","last_clean":"2023-10-06T11:41:56.518118+0000","last_became_active":"2023-10-06T11:41:56.512699+0000","last_became_peered":"2023-10-06T11:41:56.512699+0000","last_unstale":"2023-10-06T11:41:56.518118+0000","last_undegraded":"2023-10-06T11:41:56.518118+0000","last_fullsized":"2023-10-06T11:41:56.518118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:20.998002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518013+0000","last_change":"2023-10-06T11:41:56.512383+0000","last_active":"2023-10-06T11:41:56.518013+0000","last_peered":"2023-10-06T11:41:56.518013+0000","last_clean":"2023-10-06T11:41:56.518013+0000","last_became_active":"2023-10-06T11:41:56.512086+0000","last_became_peered":"2023-10-06T11:41:56.512086+0000","last_unstale":"2023-10-06T11:41:56.518013+0000","last_undegraded":"2023-10-06T11:41:56.518013+0000","last_fullsized":"2023-10-06T11:41:56.518013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:44.564348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514263+0000","last_change":"2023-10-06T11:41:56.510312+0000","last_active":"2023-10-06T11:41:56.514263+0000","last_peered":"2023-10-06T11:41:56.514263+0000","last_clean":"2023-10-06T11:41:56.514263+0000","last_became_active":"2023-10-06T11:41:56.510046+0000","last_became_peered":"2023-10-06T11:41:56.510046+0000","last_unstale":"2023-10-06T11:41:56.514263+0000","last_undegraded":"2023-10-06T11:41:56.514263+0000","last_fullsized":"2023-10-06T11:41:56.514263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522268+0000","last_change":"2023-10-06T11:41:56.518929+0000","last_active":"2023-10-06T11:41:56.522268+0000","last_peered":"2023-10-06T11:41:56.522268+0000","last_clean":"2023-10-06T11:41:56.522268+0000","last_became_active":"2023-10-06T11:41:56.518671+0000","last_became_peered":"2023-10-06T11:41:56.518671+0000","last_unstale":"2023-10-06T11:41:56.522268+0000","last_undegraded":"2023-10-06T11:41:56.522268+0000","last_fullsized":"2023-10-06T11:41:56.522268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513758+0000","last_change":"2023-10-06T11:41:56.509452+0000","last_active":"2023-10-06T11:41:56.513758+0000","last_peered":"2023-10-06T11:41:56.513758+0000","last_clean":"2023-10-06T11:41:56.513758+0000","last_became_active":"2023-10-06T11:41:56.509141+0000","last_became_peered":"2023-10-06T11:41:56.509141+0000","last_unstale":"2023-10-06T11:41:56.513758+0000","last_undegraded":"2023-10-06T11:41:56.513758+0000","last_fullsized":"2023-10-06T11:41:56.513758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515869+0000","last_change":"2023-10-06T11:41:56.511167+0000","last_active":"2023-10-06T11:41:56.515869+0000","last_peered":"2023-10-06T11:41:56.515869+0000","last_clean":"2023-10-06T11:41:56.515869+0000","last_became_active":"2023-10-06T11:41:56.510849+0000","last_became_peered":"2023-10-06T11:41:56.510849+0000","last_unstale":"2023-10-06T11:41:56.515869+0000","last_undegraded":"2023-10-06T11:41:56.515869+0000","last_fullsized":"2023-10-06T11:41:56.515869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516793+0000","last_change":"2023-10-06T11:41:56.511248+0000","last_active":"2023-10-06T11:41:56.516793+0000","last_peered":"2023-10-06T11:41:56.516793+0000","last_clean":"2023-10-06T11:41:56.516793+0000","last_became_active":"2023-10-06T11:41:56.510023+0000","last_became_peered":"2023-10-06T11:41:56.510023+0000","last_unstale":"2023-10-06T11:41:56.516793+0000","last_undegraded":"2023-10-06T11:41:56.516793+0000","last_fullsized":"2023-10-06T11:41:56.516793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519577+0000","last_change":"2023-10-06T11:41:56.513818+0000","last_active":"2023-10-06T11:41:56.519577+0000","last_peered":"2023-10-06T11:41:56.519577+0000","last_clean":"2023-10-06T11:41:56.519577+0000","last_became_active":"2023-10-06T11:41:56.513508+0000","last_became_peered":"2023-10-06T11:41:56.513508+0000","last_unstale":"2023-10-06T11:41:56.519577+0000","last_undegraded":"2023-10-06T11:41:56.519577+0000","last_fullsized":"2023-10-06T11:41:56.519577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:25.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519973+0000","last_change":"2023-10-06T11:41:56.514028+0000","last_active":"2023-10-06T11:41:56.519973+0000","last_peered":"2023-10-06T11:41:56.519973+0000","last_clean":"2023-10-06T11:41:56.519973+0000","last_became_active":"2023-10-06T11:41:56.513176+0000","last_became_peered":"2023-10-06T11:41:56.513176+0000","last_unstale":"2023-10-06T11:41:56.519973+0000","last_undegraded":"2023-10-06T11:41:56.519973+0000","last_fullsized":"2023-10-06T11:41:56.519973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516670+0000","last_change":"2023-10-06T11:41:56.513150+0000","last_active":"2023-10-06T11:41:56.516670+0000","last_peered":"2023-10-06T11:41:56.516670+0000","last_clean":"2023-10-06T11:41:56.516670+0000","last_became_active":"2023-10-06T11:41:56.512844+0000","last_became_peered":"2023-10-06T11:41:56.512844+0000","last_unstale":"2023-10-06T11:41:56.516670+0000","last_undegraded":"2023-10-06T11:41:56.516670+0000","last_fullsized":"2023-10-06T11:41:56.516670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519663+0000","last_change":"2023-10-06T11:41:56.512179+0000","last_active":"2023-10-06T11:41:56.519663+0000","last_peered":"2023-10-06T11:41:56.519663+0000","last_clean":"2023-10-06T11:41:56.519663+0000","last_became_active":"2023-10-06T11:41:56.511885+0000","last_became_peered":"2023-10-06T11:41:56.511885+0000","last_unstale":"2023-10-06T11:41:56.519663+0000","last_undegraded":"2023-10-06T11:41:56.519663+0000","last_fullsized":"2023-10-06T11:41:56.519663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520009+0000","last_change":"2023-10-06T11:41:56.514447+0000","last_active":"2023-10-06T11:41:56.520009+0000","last_peered":"2023-10-06T11:41:56.520009+0000","last_clean":"2023-10-06T11:41:56.520009+0000","last_became_active":"2023-10-06T11:41:56.514134+0000","last_became_peered":"2023-10-06T11:41:56.514134+0000","last_unstale":"2023-10-06T11:41:56.520009+0000","last_undegraded":"2023-10-06T11:41:56.520009+0000","last_fullsized":"2023-10-06T11:41:56.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516608+0000","last_change":"2023-10-06T11:41:56.513062+0000","last_active":"2023-10-06T11:41:56.516608+0000","last_peered":"2023-10-06T11:41:56.516608+0000","last_clean":"2023-10-06T11:41:56.516608+0000","last_became_active":"2023-10-06T11:41:56.512812+0000","last_became_peered":"2023-10-06T11:41:56.512812+0000","last_unstale":"2023-10-06T11:41:56.516608+0000","last_undegraded":"2023-10-06T11:41:56.516608+0000","last_fullsized":"2023-10-06T11:41:56.516608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517749+0000","last_change":"2023-10-06T11:41:56.514788+0000","last_active":"2023-10-06T11:41:56.517749+0000","last_peered":"2023-10-06T11:41:56.517749+0000","last_clean":"2023-10-06T11:41:56.517749+0000","last_became_active":"2023-10-06T11:41:56.514494+0000","last_became_peered":"2023-10-06T11:41:56.514494+0000","last_unstale":"2023-10-06T11:41:56.517749+0000","last_undegraded":"2023-10-06T11:41:56.517749+0000","last_fullsized":"2023-10-06T11:41:56.517749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516257+0000","last_change":"2023-10-06T11:41:56.511137+0000","last_active":"2023-10-06T11:41:56.516257+0000","last_peered":"2023-10-06T11:41:56.516257+0000","last_clean":"2023-10-06T11:41:56.516257+0000","last_became_active":"2023-10-06T11:41:56.510826+0000","last_became_peered":"2023-10-06T11:41:56.510826+0000","last_unstale":"2023-10-06T11:41:56.516257+0000","last_undegraded":"2023-10-06T11:41:56.516257+0000","last_fullsized":"2023-10-06T11:41:56.516257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519152+0000","last_change":"2023-10-06T11:41:56.513345+0000","last_active":"2023-10-06T11:41:56.519152+0000","last_peered":"2023-10-06T11:41:56.519152+0000","last_clean":"2023-10-06T11:41:56.519152+0000","last_became_active":"2023-10-06T11:41:56.513068+0000","last_became_peered":"2023-10-06T11:41:56.513068+0000","last_unstale":"2023-10-06T11:41:56.519152+0000","last_undegraded":"2023-10-06T11:41:56.519152+0000","last_fullsized":"2023-10-06T11:41:56.519152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.529014+0000","last_change":"2023-10-06T11:41:55.504888+0000","last_active":"2023-10-06T11:41:56.529014+0000","last_peered":"2023-10-06T11:41:56.529014+0000","last_clean":"2023-10-06T11:41:56.529014+0000","last_became_active":"2023-10-06T11:41:55.504602+0000","last_became_peered":"2023-10-06T11:41:55.504602+0000","last_unstale":"2023-10-06T11:41:56.529014+0000","last_undegraded":"2023-10-06T11:41:56.529014+0000","last_fullsized":"2023-10-06T11:41:56.529014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518587+0000","last_change":"2023-10-06T11:41:56.512500+0000","last_active":"2023-10-06T11:41:56.518587+0000","last_peered":"2023-10-06T11:41:56.518587+0000","last_clean":"2023-10-06T11:41:56.518587+0000","last_became_active":"2023-10-06T11:41:56.512213+0000","last_became_peered":"2023-10-06T11:41:56.512213+0000","last_unstale":"2023-10-06T11:41:56.518587+0000","last_undegraded":"2023-10-06T11:41:56.518587+0000","last_fullsized":"2023-10-06T11:41:56.518587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497430+0000","last_change":"2023-10-06T11:41:55.505880+0000","last_active":"2023-10-06T11:41:56.497430+0000","last_peered":"2023-10-06T11:41:56.497430+0000","last_clean":"2023-10-06T11:41:56.497430+0000","last_became_active":"2023-10-06T11:41:55.504907+0000","last_became_peered":"2023-10-06T11:41:55.504907+0000","last_unstale":"2023-10-06T11:41:56.497430+0000","last_undegraded":"2023-10-06T11:41:56.497430+0000","last_fullsized":"2023-10-06T11:41:56.497430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519692+0000","last_change":"2023-10-06T11:41:56.514094+0000","last_active":"2023-10-06T11:41:56.519692+0000","last_peered":"2023-10-06T11:41:56.519692+0000","last_clean":"2023-10-06T11:41:56.519692+0000","last_became_active":"2023-10-06T11:41:56.513833+0000","last_became_peered":"2023-10-06T11:41:56.513833+0000","last_unstale":"2023-10-06T11:41:56.519692+0000","last_undegraded":"2023-10-06T11:41:56.519692+0000","last_fullsized":"2023-10-06T11:41:56.519692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503895+0000","last_change":"2023-10-06T11:41:55.504864+0000","last_active":"2023-10-06T11:41:56.503895+0000","last_peered":"2023-10-06T11:41:56.503895+0000","last_clean":"2023-10-06T11:41:56.503895+0000","last_became_active":"2023-10-06T11:41:55.504166+0000","last_became_peered":"2023-10-06T11:41:55.504166+0000","last_unstale":"2023-10-06T11:41:56.503895+0000","last_undegraded":"2023-10-06T11:41:56.503895+0000","last_fullsized":"2023-10-06T11:41:56.503895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514214+0000","last_change":"2023-10-06T11:41:56.509697+0000","last_active":"2023-10-06T11:41:56.514214+0000","last_peered":"2023-10-06T11:41:56.514214+0000","last_clean":"2023-10-06T11:41:56.514214+0000","last_became_active":"2023-10-06T11:41:56.509414+0000","last_became_peered":"2023-10-06T11:41:56.509414+0000","last_unstale":"2023-10-06T11:41:56.514214+0000","last_undegraded":"2023-10-06T11:41:56.514214+0000","last_fullsized":"2023-10-06T11:41:56.514214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507789+0000","last_change":"2023-10-06T11:41:55.496678+0000","last_active":"2023-10-06T11:41:56.507789+0000","last_peered":"2023-10-06T11:41:56.507789+0000","last_clean":"2023-10-06T11:41:56.507789+0000","last_became_active":"2023-10-06T11:41:55.494896+0000","last_became_peered":"2023-10-06T11:41:55.494896+0000","last_unstale":"2023-10-06T11:41:56.507789+0000","last_undegraded":"2023-10-06T11:41:56.507789+0000","last_fullsized":"2023-10-06T11:41:56.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517092+0000","last_change":"2023-10-06T11:41:56.512202+0000","last_active":"2023-10-06T11:41:56.517092+0000","last_peered":"2023-10-06T11:41:56.517092+0000","last_clean":"2023-10-06T11:41:56.517092+0000","last_became_active":"2023-10-06T11:41:56.511913+0000","last_became_peered":"2023-10-06T11:41:56.511913+0000","last_unstale":"2023-10-06T11:41:56.517092+0000","last_undegraded":"2023-10-06T11:41:56.517092+0000","last_fullsized":"2023-10-06T11:41:56.517092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502121+0000","last_change":"2023-10-06T11:41:55.500940+0000","last_active":"2023-10-06T11:41:56.502121+0000","last_peered":"2023-10-06T11:41:56.502121+0000","last_clean":"2023-10-06T11:41:56.502121+0000","last_became_active":"2023-10-06T11:41:55.500652+0000","last_became_peered":"2023-10-06T11:41:55.500652+0000","last_unstale":"2023-10-06T11:41:56.502121+0000","last_undegraded":"2023-10-06T11:41:56.502121+0000","last_fullsized":"2023-10-06T11:41:56.502121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520515+0000","last_change":"2023-10-06T11:41:56.515837+0000","last_active":"2023-10-06T11:41:56.520515+0000","last_peered":"2023-10-06T11:41:56.520515+0000","last_clean":"2023-10-06T11:41:56.520515+0000","last_became_active":"2023-10-06T11:41:56.515511+0000","last_became_peered":"2023-10-06T11:41:56.515511+0000","last_unstale":"2023-10-06T11:41:56.520515+0000","last_undegraded":"2023-10-06T11:41:56.520515+0000","last_fullsized":"2023-10-06T11:41:56.520515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534860+0000","last_change":"2023-10-06T11:41:55.509167+0000","last_active":"2023-10-06T11:41:56.534860+0000","last_peered":"2023-10-06T11:41:56.534860+0000","last_clean":"2023-10-06T11:41:56.534860+0000","last_became_active":"2023-10-06T11:41:55.508968+0000","last_became_peered":"2023-10-06T11:41:55.508968+0000","last_unstale":"2023-10-06T11:41:56.534860+0000","last_undegraded":"2023-10-06T11:41:56.534860+0000","last_fullsized":"2023-10-06T11:41:56.534860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:13.549315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520107+0000","last_change":"2023-10-06T11:41:56.515835+0000","last_active":"2023-10-06T11:41:56.520107+0000","last_peered":"2023-10-06T11:41:56.520107+0000","last_clean":"2023-10-06T11:41:56.520107+0000","last_became_active":"2023-10-06T11:41:56.514755+0000","last_became_peered":"2023-10-06T11:41:56.514755+0000","last_unstale":"2023-10-06T11:41:56.520107+0000","last_undegraded":"2023-10-06T11:41:56.520107+0000","last_fullsized":"2023-10-06T11:41:56.520107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.536294+0000","last_change":"2023-10-06T11:41:55.503423+0000","last_active":"2023-10-06T11:41:56.536294+0000","last_peered":"2023-10-06T11:41:56.536294+0000","last_clean":"2023-10-06T11:41:56.536294+0000","last_became_active":"2023-10-06T11:41:55.499943+0000","last_became_peered":"2023-10-06T11:41:55.499943+0000","last_unstale":"2023-10-06T11:41:56.536294+0000","last_undegraded":"2023-10-06T11:41:56.536294+0000","last_fullsized":"2023-10-06T11:41:56.536294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519939+0000","last_change":"2023-10-06T11:41:56.513385+0000","last_active":"2023-10-06T11:41:56.519939+0000","last_peered":"2023-10-06T11:41:56.519939+0000","last_clean":"2023-10-06T11:41:56.519939+0000","last_became_active":"2023-10-06T11:41:56.513091+0000","last_became_peered":"2023-10-06T11:41:56.513091+0000","last_unstale":"2023-10-06T11:41:56.519939+0000","last_undegraded":"2023-10-06T11:41:56.519939+0000","last_fullsized":"2023-10-06T11:41:56.519939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:31.044745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534779+0000","last_change":"2023-10-06T11:41:55.508640+0000","last_active":"2023-10-06T11:41:56.534779+0000","last_peered":"2023-10-06T11:41:56.534779+0000","last_clean":"2023-10-06T11:41:56.534779+0000","last_became_active":"2023-10-06T11:41:55.508359+0000","last_became_peered":"2023-10-06T11:41:55.508359+0000","last_unstale":"2023-10-06T11:41:56.534779+0000","last_undegraded":"2023-10-06T11:41:56.534779+0000","last_fullsized":"2023-10-06T11:41:56.534779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:20.652268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520221+0000","last_change":"2023-10-06T11:41:56.518196+0000","last_active":"2023-10-06T11:41:56.520221+0000","last_peered":"2023-10-06T11:41:56.520221+0000","last_clean":"2023-10-06T11:41:56.520221+0000","last_became_active":"2023-10-06T11:41:56.517922+0000","last_became_peered":"2023-10-06T11:41:56.517922+0000","last_unstale":"2023-10-06T11:41:56.520221+0000","last_undegraded":"2023-10-06T11:41:56.520221+0000","last_fullsized":"2023-10-06T11:41:56.520221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507442+0000","last_change":"2023-10-06T11:41:55.498881+0000","last_active":"2023-10-06T11:41:56.507442+0000","last_peered":"2023-10-06T11:41:56.507442+0000","last_clean":"2023-10-06T11:41:56.507442+0000","last_became_active":"2023-10-06T11:41:55.498646+0000","last_became_peered":"2023-10-06T11:41:55.498646+0000","last_unstale":"2023-10-06T11:41:56.507442+0000","last_undegraded":"2023-10-06T11:41:56.507442+0000","last_fullsized":"2023-10-06T11:41:56.507442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.521939+0000","last_change":"2023-10-06T11:41:56.514291+0000","last_active":"2023-10-06T11:41:56.521939+0000","last_peered":"2023-10-06T11:41:56.521939+0000","last_clean":"2023-10-06T11:41:56.521939+0000","last_became_active":"2023-10-06T11:41:56.513811+0000","last_became_peered":"2023-10-06T11:41:56.513811+0000","last_unstale":"2023-10-06T11:41:56.521939+0000","last_undegraded":"2023-10-06T11:41:56.521939+0000","last_fullsized":"2023-10-06T11:41:56.521939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506360+0000","last_change":"2023-10-06T11:41:55.510456+0000","last_active":"2023-10-06T11:41:56.506360+0000","last_peered":"2023-10-06T11:41:56.506360+0000","last_clean":"2023-10-06T11:41:56.506360+0000","last_became_active":"2023-10-06T11:41:55.510148+0000","last_became_peered":"2023-10-06T11:41:55.510148+0000","last_unstale":"2023-10-06T11:41:56.506360+0000","last_undegraded":"2023-10-06T11:41:56.506360+0000","last_fullsized":"2023-10-06T11:41:56.506360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519708+0000","last_change":"2023-10-06T11:41:56.516787+0000","last_active":"2023-10-06T11:41:56.519708+0000","last_peered":"2023-10-06T11:41:56.519708+0000","last_clean":"2023-10-06T11:41:56.519708+0000","last_became_active":"2023-10-06T11:41:56.516510+0000","last_became_peered":"2023-10-06T11:41:56.516510+0000","last_unstale":"2023-10-06T11:41:56.519708+0000","last_undegraded":"2023-10-06T11:41:56.519708+0000","last_fullsized":"2023-10-06T11:41:56.519708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.532331+0000","last_change":"2023-10-06T11:41:55.502794+0000","last_active":"2023-10-06T11:41:56.532331+0000","last_peered":"2023-10-06T11:41:56.532331+0000","last_clean":"2023-10-06T11:41:56.532331+0000","last_became_active":"2023-10-06T11:41:55.502522+0000","last_became_peered":"2023-10-06T11:41:55.502522+0000","last_unstale":"2023-10-06T11:41:56.532331+0000","last_undegraded":"2023-10-06T11:41:56.532331+0000","last_fullsized":"2023-10-06T11:41:56.532331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520360+0000","last_change":"2023-10-06T11:41:56.515679+0000","last_active":"2023-10-06T11:41:56.520360+0000","last_peered":"2023-10-06T11:41:56.520360+0000","last_clean":"2023-10-06T11:41:56.520360+0000","last_became_active":"2023-10-06T11:41:56.515167+0000","last_became_peered":"2023-10-06T11:41:56.515167+0000","last_unstale":"2023-10-06T11:41:56.520360+0000","last_undegraded":"2023-10-06T11:41:56.520360+0000","last_fullsized":"2023-10-06T11:41:56.520360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534434+0000","last_change":"2023-10-06T11:41:55.506331+0000","last_active":"2023-10-06T11:41:56.534434+0000","last_peered":"2023-10-06T11:41:56.534434+0000","last_clean":"2023-10-06T11:41:56.534434+0000","last_became_active":"2023-10-06T11:41:55.506044+0000","last_became_peered":"2023-10-06T11:41:55.506044+0000","last_unstale":"2023-10-06T11:41:56.534434+0000","last_undegraded":"2023-10-06T11:41:56.534434+0000","last_fullsized":"2023-10-06T11:41:56.534434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514023+0000","last_change":"2023-10-06T11:41:56.509291+0000","last_active":"2023-10-06T11:41:56.514023+0000","last_peered":"2023-10-06T11:41:56.514023+0000","last_clean":"2023-10-06T11:41:56.514023+0000","last_became_active":"2023-10-06T11:41:56.509004+0000","last_became_peered":"2023-10-06T11:41:56.509004+0000","last_unstale":"2023-10-06T11:41:56.514023+0000","last_undegraded":"2023-10-06T11:41:56.514023+0000","last_fullsized":"2023-10-06T11:41:56.514023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534956+0000","last_change":"2023-10-06T11:41:55.510357+0000","last_active":"2023-10-06T11:41:56.534956+0000","last_peered":"2023-10-06T11:41:56.534956+0000","last_clean":"2023-10-06T11:41:56.534956+0000","last_became_active":"2023-10-06T11:41:55.510053+0000","last_became_peered":"2023-10-06T11:41:55.510053+0000","last_unstale":"2023-10-06T11:41:56.534956+0000","last_undegraded":"2023-10-06T11:41:56.534956+0000","last_fullsized":"2023-10-06T11:41:56.534956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516748+0000","last_change":"2023-10-06T11:41:56.510920+0000","last_active":"2023-10-06T11:41:56.516748+0000","last_peered":"2023-10-06T11:41:56.516748+0000","last_clean":"2023-10-06T11:41:56.516748+0000","last_became_active":"2023-10-06T11:41:56.510620+0000","last_became_peered":"2023-10-06T11:41:56.510620+0000","last_unstale":"2023-10-06T11:41:56.516748+0000","last_undegraded":"2023-10-06T11:41:56.516748+0000","last_fullsized":"2023-10-06T11:41:56.516748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:01.202344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534773+0000","last_change":"2023-10-06T11:41:55.501172+0000","last_active":"2023-10-06T11:41:56.534773+0000","last_peered":"2023-10-06T11:41:56.534773+0000","last_clean":"2023-10-06T11:41:56.534773+0000","last_became_active":"2023-10-06T11:41:55.500891+0000","last_became_peered":"2023-10-06T11:41:55.500891+0000","last_unstale":"2023-10-06T11:41:56.534773+0000","last_undegraded":"2023-10-06T11:41:56.534773+0000","last_fullsized":"2023-10-06T11:41:56.534773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:31.984593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516634+0000","last_change":"2023-10-06T11:41:56.512054+0000","last_active":"2023-10-06T11:41:56.516634+0000","last_peered":"2023-10-06T11:41:56.516634+0000","last_clean":"2023-10-06T11:41:56.516634+0000","last_became_active":"2023-10-06T11:41:56.511385+0000","last_became_peered":"2023-10-06T11:41:56.511385+0000","last_unstale":"2023-10-06T11:41:56.516634+0000","last_undegraded":"2023-10-06T11:41:56.516634+0000","last_fullsized":"2023-10-06T11:41:56.516634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.394264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.501453+0000","last_change":"2023-10-06T11:41:55.505446+0000","last_active":"2023-10-06T11:41:56.501453+0000","last_peered":"2023-10-06T11:41:56.501453+0000","last_clean":"2023-10-06T11:41:56.501453+0000","last_became_active":"2023-10-06T11:41:55.505189+0000","last_became_peered":"2023-10-06T11:41:55.505189+0000","last_unstale":"2023-10-06T11:41:56.501453+0000","last_undegraded":"2023-10-06T11:41:56.501453+0000","last_fullsized":"2023-10-06T11:41:56.501453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519776+0000","last_change":"2023-10-06T11:41:56.517521+0000","last_active":"2023-10-06T11:41:56.519776+0000","last_peered":"2023-10-06T11:41:56.519776+0000","last_clean":"2023-10-06T11:41:56.519776+0000","last_became_active":"2023-10-06T11:41:56.517084+0000","last_became_peered":"2023-10-06T11:41:56.517084+0000","last_unstale":"2023-10-06T11:41:56.519776+0000","last_undegraded":"2023-10-06T11:41:56.519776+0000","last_fullsized":"2023-10-06T11:41:56.519776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531354+0000","last_change":"2023-10-06T11:41:55.509060+0000","last_active":"2023-10-06T11:41:56.531354+0000","last_peered":"2023-10-06T11:41:56.531354+0000","last_clean":"2023-10-06T11:41:56.531354+0000","last_became_active":"2023-10-06T11:41:55.508766+0000","last_became_peered":"2023-10-06T11:41:55.508766+0000","last_unstale":"2023-10-06T11:41:56.531354+0000","last_undegraded":"2023-10-06T11:41:56.531354+0000","last_fullsized":"2023-10-06T11:41:56.531354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514111+0000","last_change":"2023-10-06T11:41:56.510155+0000","last_active":"2023-10-06T11:41:56.514111+0000","last_peered":"2023-10-06T11:41:56.514111+0000","last_clean":"2023-10-06T11:41:56.514111+0000","last_became_active":"2023-10-06T11:41:56.509859+0000","last_became_peered":"2023-10-06T11:41:56.509859+0000","last_unstale":"2023-10-06T11:41:56.514111+0000","last_undegraded":"2023-10-06T11:41:56.514111+0000","last_fullsized":"2023-10-06T11:41:56.514111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507800+0000","last_change":"2023-10-06T11:41:55.511135+0000","last_active":"2023-10-06T11:41:56.507800+0000","last_peered":"2023-10-06T11:41:56.507800+0000","last_clean":"2023-10-06T11:41:56.507800+0000","last_became_active":"2023-10-06T11:41:55.510802+0000","last_became_peered":"2023-10-06T11:41:55.510802+0000","last_unstale":"2023-10-06T11:41:56.507800+0000","last_undegraded":"2023-10-06T11:41:56.507800+0000","last_fullsized":"2023-10-06T11:41:56.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516764+0000","last_change":"2023-10-06T11:41:56.510930+0000","last_active":"2023-10-06T11:41:56.516764+0000","last_peered":"2023-10-06T11:41:56.516764+0000","last_clean":"2023-10-06T11:41:56.516764+0000","last_became_active":"2023-10-06T11:41:56.510315+0000","last_became_peered":"2023-10-06T11:41:56.510315+0000","last_unstale":"2023-10-06T11:41:56.516764+0000","last_undegraded":"2023-10-06T11:41:56.516764+0000","last_fullsized":"2023-10-06T11:41:56.516764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500145+0000","last_change":"2023-10-06T11:41:55.506086+0000","last_active":"2023-10-06T11:41:56.500145+0000","last_peered":"2023-10-06T11:41:56.500145+0000","last_clean":"2023-10-06T11:41:56.500145+0000","last_became_active":"2023-10-06T11:41:55.505822+0000","last_became_peered":"2023-10-06T11:41:55.505822+0000","last_unstale":"2023-10-06T11:41:56.500145+0000","last_undegraded":"2023-10-06T11:41:56.500145+0000","last_fullsized":"2023-10-06T11:41:56.500145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520047+0000","last_change":"2023-10-06T11:41:56.514492+0000","last_active":"2023-10-06T11:41:56.520047+0000","last_peered":"2023-10-06T11:41:56.520047+0000","last_clean":"2023-10-06T11:41:56.520047+0000","last_became_active":"2023-10-06T11:41:56.513778+0000","last_became_peered":"2023-10-06T11:41:56.513778+0000","last_unstale":"2023-10-06T11:41:56.520047+0000","last_undegraded":"2023-10-06T11:41:56.520047+0000","last_fullsized":"2023-10-06T11:41:56.520047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503812+0000","last_change":"2023-10-06T11:41:55.505373+0000","last_active":"2023-10-06T11:41:56.503812+0000","last_peered":"2023-10-06T11:41:56.503812+0000","last_clean":"2023-10-06T11:41:56.503812+0000","last_became_active":"2023-10-06T11:41:55.505092+0000","last_became_peered":"2023-10-06T11:41:55.505092+0000","last_unstale":"2023-10-06T11:41:56.503812+0000","last_undegraded":"2023-10-06T11:41:56.503812+0000","last_fullsized":"2023-10-06T11:41:56.503812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516491+0000","last_change":"2023-10-06T11:41:56.512965+0000","last_active":"2023-10-06T11:41:56.516491+0000","last_peered":"2023-10-06T11:41:56.516491+0000","last_clean":"2023-10-06T11:41:56.516491+0000","last_became_active":"2023-10-06T11:41:56.512684+0000","last_became_peered":"2023-10-06T11:41:56.512684+0000","last_unstale":"2023-10-06T11:41:56.516491+0000","last_undegraded":"2023-10-06T11:41:56.516491+0000","last_fullsized":"2023-10-06T11:41:56.516491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534650+0000","last_change":"2023-10-06T11:41:55.510312+0000","last_active":"2023-10-06T11:41:56.534650+0000","last_peered":"2023-10-06T11:41:56.534650+0000","last_clean":"2023-10-06T11:41:56.534650+0000","last_became_active":"2023-10-06T11:41:55.510044+0000","last_became_peered":"2023-10-06T11:41:55.510044+0000","last_unstale":"2023-10-06T11:41:56.534650+0000","last_undegraded":"2023-10-06T11:41:56.534650+0000","last_fullsized":"2023-10-06T11:41:56.534650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500088+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:41:56.500088+0000","last_peered":"2023-10-06T11:41:56.500088+0000","last_clean":"2023-10-06T11:41:56.500088+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:41:56.500088+0000","last_undegraded":"2023-10-06T11:41:56.500088+0000","last_fullsized":"2023-10-06T11:41:56.500088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519625+0000","last_change":"2023-10-06T11:41:56.515565+0000","last_active":"2023-10-06T11:41:56.519625+0000","last_peered":"2023-10-06T11:41:56.519625+0000","last_clean":"2023-10-06T11:41:56.519625+0000","last_became_active":"2023-10-06T11:41:56.515007+0000","last_became_peered":"2023-10-06T11:41:56.515007+0000","last_unstale":"2023-10-06T11:41:56.519625+0000","last_undegraded":"2023-10-06T11:41:56.519625+0000","last_fullsized":"2023-10-06T11:41:56.519625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503231+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:41:56.503231+0000","last_peered":"2023-10-06T11:41:56.503231+0000","last_clean":"2023-10-06T11:41:56.503231+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:41:56.503231+0000","last_undegraded":"2023-10-06T11:41:56.503231+0000","last_fullsized":"2023-10-06T11:41:56.503231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507965+0000","last_change":"2023-10-06T11:41:55.500074+0000","last_active":"2023-10-06T11:41:56.507965+0000","last_peered":"2023-10-06T11:41:56.507965+0000","last_clean":"2023-10-06T11:41:56.507965+0000","last_became_active":"2023-10-06T11:41:55.499784+0000","last_became_peered":"2023-10-06T11:41:55.499784+0000","last_unstale":"2023-10-06T11:41:56.507965+0000","last_undegraded":"2023-10-06T11:41:56.507965+0000","last_fullsized":"2023-10-06T11:41:56.507965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513641+0000","last_change":"2023-10-06T11:41:56.509030+0000","last_active":"2023-10-06T11:41:56.513641+0000","last_peered":"2023-10-06T11:41:56.513641+0000","last_clean":"2023-10-06T11:41:56.513641+0000","last_became_active":"2023-10-06T11:41:56.508720+0000","last_became_peered":"2023-10-06T11:41:56.508720+0000","last_unstale":"2023-10-06T11:41:56.513641+0000","last_undegraded":"2023-10-06T11:41:56.513641+0000","last_fullsized":"2023-10-06T11:41:56.513641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:51.155604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497934+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:41:56.497934+0000","last_peered":"2023-10-06T11:41:56.497934+0000","last_clean":"2023-10-06T11:41:56.497934+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:41:56.497934+0000","last_undegraded":"2023-10-06T11:41:56.497934+0000","last_fullsized":"2023-10-06T11:41:56.497934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506182+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:41:56.506182+0000","last_peered":"2023-10-06T11:41:56.506182+0000","last_clean":"2023-10-06T11:41:56.506182+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:41:56.506182+0000","last_undegraded":"2023-10-06T11:41:56.506182+0000","last_fullsized":"2023-10-06T11:41:56.506182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507640+0000","last_change":"2023-10-06T11:41:55.506608+0000","last_active":"2023-10-06T11:41:56.507640+0000","last_peered":"2023-10-06T11:41:56.507640+0000","last_clean":"2023-10-06T11:41:56.507640+0000","last_became_active":"2023-10-06T11:41:55.506234+0000","last_became_peered":"2023-10-06T11:41:55.506234+0000","last_unstale":"2023-10-06T11:41:56.507640+0000","last_undegraded":"2023-10-06T11:41:56.507640+0000","last_fullsized":"2023-10-06T11:41:56.507640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519724+0000","last_change":"2023-10-06T11:41:56.516915+0000","last_active":"2023-10-06T11:41:56.519724+0000","last_peered":"2023-10-06T11:41:56.519724+0000","last_clean":"2023-10-06T11:41:56.519724+0000","last_became_active":"2023-10-06T11:41:56.516184+0000","last_became_peered":"2023-10-06T11:41:56.516184+0000","last_unstale":"2023-10-06T11:41:56.519724+0000","last_undegraded":"2023-10-06T11:41:56.519724+0000","last_fullsized":"2023-10-06T11:41:56.519724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505779+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:41:56.505779+0000","last_peered":"2023-10-06T11:41:56.505779+0000","last_clean":"2023-10-06T11:41:56.505779+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:41:56.505779+0000","last_undegraded":"2023-10-06T11:41:56.505779+0000","last_fullsized":"2023-10-06T11:41:56.505779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497472+0000","last_change":"2023-10-06T11:41:55.506060+0000","last_active":"2023-10-06T11:41:56.497472+0000","last_peered":"2023-10-06T11:41:56.497472+0000","last_clean":"2023-10-06T11:41:56.497472+0000","last_became_active":"2023-10-06T11:41:55.505551+0000","last_became_peered":"2023-10-06T11:41:55.505551+0000","last_unstale":"2023-10-06T11:41:56.497472+0000","last_undegraded":"2023-10-06T11:41:56.497472+0000","last_fullsized":"2023-10-06T11:41:56.497472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:23.340598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515790+0000","last_change":"2023-10-06T11:41:56.509141+0000","last_active":"2023-10-06T11:41:56.515790+0000","last_peered":"2023-10-06T11:41:56.515790+0000","last_clean":"2023-10-06T11:41:56.515790+0000","last_became_active":"2023-10-06T11:41:56.508814+0000","last_became_peered":"2023-10-06T11:41:56.508814+0000","last_unstale":"2023-10-06T11:41:56.515790+0000","last_undegraded":"2023-10-06T11:41:56.515790+0000","last_fullsized":"2023-10-06T11:41:56.515790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505943+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:41:56.505943+0000","last_peered":"2023-10-06T11:41:56.505943+0000","last_clean":"2023-10-06T11:41:56.505943+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:41:56.505943+0000","last_undegraded":"2023-10-06T11:41:56.505943+0000","last_fullsized":"2023-10-06T11:41:56.505943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502767+0000","last_change":"2023-10-06T11:41:55.504016+0000","last_active":"2023-10-06T11:41:56.502767+0000","last_peered":"2023-10-06T11:41:56.502767+0000","last_clean":"2023-10-06T11:41:56.502767+0000","last_became_active":"2023-10-06T11:41:55.503831+0000","last_became_peered":"2023-10-06T11:41:55.503831+0000","last_unstale":"2023-10-06T11:41:56.502767+0000","last_undegraded":"2023-10-06T11:41:56.502767+0000","last_fullsized":"2023-10-06T11:41:56.502767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519487+0000","last_change":"2023-10-06T11:41:56.515759+0000","last_active":"2023-10-06T11:41:56.519487+0000","last_peered":"2023-10-06T11:41:56.519487+0000","last_clean":"2023-10-06T11:41:56.519487+0000","last_became_active":"2023-10-06T11:41:56.514780+0000","last_became_peered":"2023-10-06T11:41:56.514780+0000","last_unstale":"2023-10-06T11:41:56.519487+0000","last_undegraded":"2023-10-06T11:41:56.519487+0000","last_fullsized":"2023-10-06T11:41:56.519487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502819+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:41:56.502819+0000","last_peered":"2023-10-06T11:41:56.502819+0000","last_clean":"2023-10-06T11:41:56.502819+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:41:56.502819+0000","last_undegraded":"2023-10-06T11:41:56.502819+0000","last_fullsized":"2023-10-06T11:41:56.502819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500296+0000","last_change":"2023-10-06T11:41:55.508163+0000","last_active":"2023-10-06T11:41:56.500296+0000","last_peered":"2023-10-06T11:41:56.500296+0000","last_clean":"2023-10-06T11:41:56.500296+0000","last_became_active":"2023-10-06T11:41:55.507855+0000","last_became_peered":"2023-10-06T11:41:55.507855+0000","last_unstale":"2023-10-06T11:41:56.500296+0000","last_undegraded":"2023-10-06T11:41:56.500296+0000","last_fullsized":"2023-10-06T11:41:56.500296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516585+0000","last_change":"2023-10-06T11:41:56.512837+0000","last_active":"2023-10-06T11:41:56.516585+0000","last_peered":"2023-10-06T11:41:56.516585+0000","last_clean":"2023-10-06T11:41:56.516585+0000","last_became_active":"2023-10-06T11:41:56.511727+0000","last_became_peered":"2023-10-06T11:41:56.511727+0000","last_unstale":"2023-10-06T11:41:56.516585+0000","last_undegraded":"2023-10-06T11:41:56.516585+0000","last_fullsized":"2023-10-06T11:41:56.516585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507674+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:41:56.507674+0000","last_peered":"2023-10-06T11:41:56.507674+0000","last_clean":"2023-10-06T11:41:56.507674+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:41:56.507674+0000","last_undegraded":"2023-10-06T11:41:56.507674+0000","last_fullsized":"2023-10-06T11:41:56.507674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497703+0000","last_change":"2023-10-06T11:41:55.508524+0000","last_active":"2023-10-06T11:41:56.497703+0000","last_peered":"2023-10-06T11:41:56.497703+0000","last_clean":"2023-10-06T11:41:56.497703+0000","last_became_active":"2023-10-06T11:41:55.505590+0000","last_became_peered":"2023-10-06T11:41:55.505590+0000","last_unstale":"2023-10-06T11:41:56.497703+0000","last_undegraded":"2023-10-06T11:41:56.497703+0000","last_fullsized":"2023-10-06T11:41:56.497703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.511544+0000","last_change":"2023-10-06T11:41:56.509540+0000","last_active":"2023-10-06T11:41:56.511544+0000","last_peered":"2023-10-06T11:41:56.511544+0000","last_clean":"2023-10-06T11:41:56.511544+0000","last_became_active":"2023-10-06T11:41:56.509208+0000","last_became_peered":"2023-10-06T11:41:56.509208+0000","last_unstale":"2023-10-06T11:41:56.511544+0000","last_undegraded":"2023-10-06T11:41:56.511544+0000","last_fullsized":"2023-10-06T11:41:56.511544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507693+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:41:56.507693+0000","last_peered":"2023-10-06T11:41:56.507693+0000","last_clean":"2023-10-06T11:41:56.507693+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:41:56.507693+0000","last_undegraded":"2023-10-06T11:41:56.507693+0000","last_fullsized":"2023-10-06T11:41:56.507693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503179+0000","last_change":"2023-10-06T11:41:55.505392+0000","last_active":"2023-10-06T11:41:56.503179+0000","last_peered":"2023-10-06T11:41:56.503179+0000","last_clean":"2023-10-06T11:41:56.503179+0000","last_became_active":"2023-10-06T11:41:55.505103+0000","last_became_peered":"2023-10-06T11:41:55.505103+0000","last_unstale":"2023-10-06T11:41:56.503179+0000","last_undegraded":"2023-10-06T11:41:56.503179+0000","last_fullsized":"2023-10-06T11:41:56.503179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516182+0000","last_change":"2023-10-06T11:41:56.509787+0000","last_active":"2023-10-06T11:41:56.516182+0000","last_peered":"2023-10-06T11:41:56.516182+0000","last_clean":"2023-10-06T11:41:56.516182+0000","last_became_active":"2023-10-06T11:41:56.509495+0000","last_became_peered":"2023-10-06T11:41:56.509495+0000","last_unstale":"2023-10-06T11:41:56.516182+0000","last_undegraded":"2023-10-06T11:41:56.516182+0000","last_fullsized":"2023-10-06T11:41:56.516182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531881+0000","last_change":"2023-10-06T11:41:55.501475+0000","last_active":"2023-10-06T11:41:56.531881+0000","last_peered":"2023-10-06T11:41:56.531881+0000","last_clean":"2023-10-06T11:41:56.531881+0000","last_became_active":"2023-10-06T11:41:55.500531+0000","last_became_peered":"2023-10-06T11:41:55.500531+0000","last_unstale":"2023-10-06T11:41:56.531881+0000","last_undegraded":"2023-10-06T11:41:56.531881+0000","last_fullsized":"2023-10-06T11:41:56.531881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519814+0000","last_change":"2023-10-06T11:41:56.518985+0000","last_active":"2023-10-06T11:41:56.519814+0000","last_peered":"2023-10-06T11:41:56.519814+0000","last_clean":"2023-10-06T11:41:56.519814+0000","last_became_active":"2023-10-06T11:41:56.518656+0000","last_became_peered":"2023-10-06T11:41:56.518656+0000","last_unstale":"2023-10-06T11:41:56.519814+0000","last_undegraded":"2023-10-06T11:41:56.519814+0000","last_fullsized":"2023-10-06T11:41:56.519814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:21.108193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534877+0000","last_change":"2023-10-06T11:41:55.506318+0000","last_active":"2023-10-06T11:41:56.534877+0000","last_peered":"2023-10-06T11:41:56.534877+0000","last_clean":"2023-10-06T11:41:56.534877+0000","last_became_active":"2023-10-06T11:41:55.506045+0000","last_became_peered":"2023-10-06T11:41:55.506045+0000","last_unstale":"2023-10-06T11:41:56.534877+0000","last_undegraded":"2023-10-06T11:41:56.534877+0000","last_fullsized":"2023-10-06T11:41:56.534877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520066+0000","last_change":"2023-10-06T11:41:56.515883+0000","last_active":"2023-10-06T11:41:56.520066+0000","last_peered":"2023-10-06T11:41:56.520066+0000","last_clean":"2023-10-06T11:41:56.520066+0000","last_became_active":"2023-10-06T11:41:56.515622+0000","last_became_peered":"2023-10-06T11:41:56.515622+0000","last_unstale":"2023-10-06T11:41:56.520066+0000","last_undegraded":"2023-10-06T11:41:56.520066+0000","last_fullsized":"2023-10-06T11:41:56.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:31.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.535054+0000","last_change":"2023-10-06T11:41:55.507981+0000","last_active":"2023-10-06T11:41:56.535054+0000","last_peered":"2023-10-06T11:41:56.535054+0000","last_clean":"2023-10-06T11:41:56.535054+0000","last_became_active":"2023-10-06T11:41:55.507665+0000","last_became_peered":"2023-10-06T11:41:55.507665+0000","last_unstale":"2023-10-06T11:41:56.535054+0000","last_undegraded":"2023-10-06T11:41:56.535054+0000","last_fullsized":"2023-10-06T11:41:56.535054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513856+0000","last_change":"2023-10-06T11:41:56.510088+0000","last_active":"2023-10-06T11:41:56.513856+0000","last_peered":"2023-10-06T11:41:56.513856+0000","last_clean":"2023-10-06T11:41:56.513856+0000","last_became_active":"2023-10-06T11:41:56.509600+0000","last_became_peered":"2023-10-06T11:41:56.509600+0000","last_unstale":"2023-10-06T11:41:56.513856+0000","last_undegraded":"2023-10-06T11:41:56.513856+0000","last_fullsized":"2023-10-06T11:41:56.513856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:16.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534276+0000","last_change":"2023-10-06T11:41:55.503624+0000","last_active":"2023-10-06T11:41:56.534276+0000","last_peered":"2023-10-06T11:41:56.534276+0000","last_clean":"2023-10-06T11:41:56.534276+0000","last_became_active":"2023-10-06T11:41:55.503318+0000","last_became_peered":"2023-10-06T11:41:55.503318+0000","last_unstale":"2023-10-06T11:41:56.534276+0000","last_undegraded":"2023-10-06T11:41:56.534276+0000","last_fullsized":"2023-10-06T11:41:56.534276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:44.190608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519991+0000","last_change":"2023-10-06T11:41:56.510835+0000","last_active":"2023-10-06T11:41:56.519991+0000","last_peered":"2023-10-06T11:41:56.519991+0000","last_clean":"2023-10-06T11:41:56.519991+0000","last_became_active":"2023-10-06T11:41:56.510547+0000","last_became_peered":"2023-10-06T11:41:56.510547+0000","last_unstale":"2023-10-06T11:41:56.519991+0000","last_undegraded":"2023-10-06T11:41:56.519991+0000","last_fullsized":"2023-10-06T11:41:56.519991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534863+0000","last_change":"2023-10-06T11:41:55.497178+0000","last_active":"2023-10-06T11:41:56.534863+0000","last_peered":"2023-10-06T11:41:56.534863+0000","last_clean":"2023-10-06T11:41:56.534863+0000","last_became_active":"2023-10-06T11:41:55.495336+0000","last_became_peered":"2023-10-06T11:41:55.495336+0000","last_unstale":"2023-10-06T11:41:56.534863+0000","last_undegraded":"2023-10-06T11:41:56.534863+0000","last_fullsized":"2023-10-06T11:41:56.534863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519888+0000","last_change":"2023-10-06T11:41:56.511788+0000","last_active":"2023-10-06T11:41:56.519888+0000","last_peered":"2023-10-06T11:41:56.519888+0000","last_clean":"2023-10-06T11:41:56.519888+0000","last_became_active":"2023-10-06T11:41:56.511488+0000","last_became_peered":"2023-10-06T11:41:56.511488+0000","last_unstale":"2023-10-06T11:41:56.519888+0000","last_undegraded":"2023-10-06T11:41:56.519888+0000","last_fullsized":"2023-10-06T11:41:56.519888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516639+0000","last_change":"2023-10-06T11:41:56.511919+0000","last_active":"2023-10-06T11:41:56.516639+0000","last_peered":"2023-10-06T11:41:56.516639+0000","last_clean":"2023-10-06T11:41:56.516639+0000","last_became_active":"2023-10-06T11:41:56.511644+0000","last_became_peered":"2023-10-06T11:41:56.511644+0000","last_unstale":"2023-10-06T11:41:56.516639+0000","last_undegraded":"2023-10-06T11:41:56.516639+0000","last_fullsized":"2023-10-06T11:41:56.516639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520154+0000","last_change":"2023-10-06T11:41:56.516583+0000","last_active":"2023-10-06T11:41:56.520154+0000","last_peered":"2023-10-06T11:41:56.520154+0000","last_clean":"2023-10-06T11:41:56.520154+0000","last_became_active":"2023-10-06T11:41:56.516241+0000","last_became_peered":"2023-10-06T11:41:56.516241+0000","last_unstale":"2023-10-06T11:41:56.520154+0000","last_undegraded":"2023-10-06T11:41:56.520154+0000","last_fullsized":"2023-10-06T11:41:56.520154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520023+0000","last_change":"2023-10-06T11:41:56.514199+0000","last_active":"2023-10-06T11:41:56.520023+0000","last_peered":"2023-10-06T11:41:56.520023+0000","last_clean":"2023-10-06T11:41:56.520023+0000","last_became_active":"2023-10-06T11:41:56.513961+0000","last_became_peered":"2023-10-06T11:41:56.513961+0000","last_unstale":"2023-10-06T11:41:56.520023+0000","last_undegraded":"2023-10-06T11:41:56.520023+0000","last_fullsized":"2023-10-06T11:41:56.520023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520267+0000","last_change":"2023-10-06T11:41:56.518788+0000","last_active":"2023-10-06T11:41:56.520267+0000","last_peered":"2023-10-06T11:41:56.520267+0000","last_clean":"2023-10-06T11:41:56.520267+0000","last_became_active":"2023-10-06T11:41:56.516640+0000","last_became_peered":"2023-10-06T11:41:56.516640+0000","last_unstale":"2023-10-06T11:41:56.520267+0000","last_undegraded":"2023-10-06T11:41:56.520267+0000","last_fullsized":"2023-10-06T11:41:56.520267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520067+0000","last_change":"2023-10-06T11:41:56.516139+0000","last_active":"2023-10-06T11:41:56.520067+0000","last_peered":"2023-10-06T11:41:56.520067+0000","last_clean":"2023-10-06T11:41:56.520067+0000","last_became_active":"2023-10-06T11:41:56.514921+0000","last_became_peered":"2023-10-06T11:41:56.514921+0000","last_unstale":"2023-10-06T11:41:56.520067+0000","last_undegraded":"2023-10-06T11:41:56.520067+0000","last_fullsized":"2023-10-06T11:41:56.520067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":671324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":71675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:00.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:00.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.825+0000 7fab362df700 1 Processor -- start 2023-10-06T11:42:00.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.825+0000 7fab362df700 1 -- start start 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab362df700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157750 0x7fab300ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab362df700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab300ce970 0x7fab300b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab362df700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab300cdc90 con 0x7fab30157380 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab362df700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab300cee10 con 0x7fab300ce970 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab362df700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300ad920 con 0x7fab30157750 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab2ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fab30157380 0x7fab300b3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56902/0 (socket says 172.21.15.16:56902) 2023-10-06T11:42:00.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab3487c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fab300ce970 0x7fab300b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.826+0000 7fab2ffff700 1 -- 172.21.15.16:0/1438437233 learned_addr learned my addr 172.21.15.16:0/1438437233 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab2f7fe700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157750 0x7fab300ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab3487c700 1 -- 172.21.15.16:0/1438437233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157750 msgr2=0x7fab300ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab3487c700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157750 0x7fab300ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab3487c700 1 -- 172.21.15.16:0/1438437233 >> v1:172.21.15.16:6789/0 conn(0x7fab30157380 legacy=0x7fab300b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab3487c700 1 -- 172.21.15.16:0/1438437233 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab300ce120 con 0x7fab300ce970 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab2effd700 1 -- 172.21.15.16:0/1438437233 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816329683 0 0) 0x7fab300cdc90 con 0x7fab30157380 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab3487c700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab300b1330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e60 tx=0x7fab28009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d9c9eff14a92b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab2effd700 1 -- 172.21.15.16:0/1438437233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800e070 con 0x7fab300ce970 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab2effd700 1 -- 172.21.15.16:0/1438437233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2800bc30 con 0x7fab300ce970 2023-10-06T11:42:00.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab2effd700 1 -- 172.21.15.16:0/1438437233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28012430 con 0x7fab300ce970 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab362df700 1 -- 172.21.15.16:0/1438437233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 msgr2=0x7fab300b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.827+0000 7fab362df700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab300b1330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab28009e60 tx=0x7fab28009780 comp rx=0 tx=0).stop 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 -- 172.21.15.16:0/1438437233 shutdown_connections 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab300b1330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 --2- 172.21.15.16:0/1438437233 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157750 0x7fab300ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 -- 172.21.15.16:0/1438437233 >> 172.21.15.16:0/1438437233 conn(0x7fab30067e60 msgr2=0x7fab30068260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 -- 172.21.15.16:0/1438437233 shutdown_connections 2023-10-06T11:42:00.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.828+0000 7fab362df700 1 -- 172.21.15.16:0/1438437233 wait complete. 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.829+0000 7fab362df700 1 Processor -- start 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.829+0000 7fab362df700 1 -- start start 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157380 0x7fab3015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab30157750 0x7fab301670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30167fb0 con 0x7fab300ce970 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c3330 con 0x7fab30157750 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c35d0 con 0x7fab30157380 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42980/0 (socket says 172.21.15.16:42980) 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 -- 172.21.15.16:0/3359130974 learned_addr learned my addr 172.21.15.16:0/3359130974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2f7fe700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157380 0x7fab3015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab3487c700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab30157750 0x7fab301670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 -- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157380 msgr2=0x7fab3015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157380 0x7fab3015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 -- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab30157750 msgr2=0x7fab301670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:00.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab30157750 0x7fab301670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:00.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 -- 172.21.15.16:0/3359130974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009a20 con 0x7fab300ce970 2023-10-06T11:42:00.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab2ffff700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c7e0 tx=0x7fab18007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab18007300 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab18007460 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab300c38b0 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab180057f0 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.830+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3015af20 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.831+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3004c380 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.832+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab18005c00 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.832+0000 7fab27fff700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 0x7fab100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.832+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab180841d0 con 0x7fab300ce970 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.832+0000 7fab2f7fe700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 0x7fab100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:00.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.833+0000 7fab2f7fe700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 0x7fab100693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab301596d0 tx=0x7fab20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:00.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:00.837+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab18051360 con 0x7fab300ce970 2023-10-06T11:42:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.024+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab300cd080 con 0x7fab10066f30 2023-10-06T11:42:01.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.032+0000 7fab27fff700 1 -- 172.21.15.16:0/3359130974 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286970 (secure 0 0 0) 0x7fab300cd080 con 0x7fab10066f30 2023-10-06T11:42:01.037 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 msgr2=0x7fab100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 0x7fab100693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fab301596d0 tx=0x7fab20008040 comp rx=0 tx=0).stop 2023-10-06T11:42:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 msgr2=0x7fab301518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab1800c7e0 tx=0x7fab18007c80 comp rx=0 tx=0).stop 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 shutdown_connections 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab300ce970 0x7fab301518b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab10066f30 0x7fab100693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fab30157750 0x7fab301670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 --2- 172.21.15.16:0/3359130974 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fab30157380 0x7fab3015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 >> 172.21.15.16:0/3359130974 conn(0x7fab30067e60 msgr2=0x7fab300b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 shutdown_connections 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.035+0000 7fab362df700 1 -- 172.21.15.16:0/3359130974 wait complete. 2023-10-06T11:42:01.042 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-06T11:42:00.060721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381188,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593532,"statfs":{"total":773094113280,"available":770655776768,"internally_reserved":0,"allocated":2953216,"data_stored":1802764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.577239"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517366+0000","last_change":"2023-10-06T11:41:56.510681+0000","last_active":"2023-10-06T11:41:56.517366+0000","last_peered":"2023-10-06T11:41:56.517366+0000","last_clean":"2023-10-06T11:41:56.517366+0000","last_became_active":"2023-10-06T11:41:56.510274+0000","last_became_peered":"2023-10-06T11:41:56.510274+0000","last_unstale":"2023-10-06T11:41:56.517366+0000","last_undegraded":"2023-10-06T11:41:56.517366+0000","last_fullsized":"2023-10-06T11:41:56.517366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:25.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519077+0000","last_change":"2023-10-06T11:41:56.512482+0000","last_active":"2023-10-06T11:41:56.519077+0000","last_peered":"2023-10-06T11:41:56.519077+0000","last_clean":"2023-10-06T11:41:56.519077+0000","last_became_active":"2023-10-06T11:41:56.512222+0000","last_became_peered":"2023-10-06T11:41:56.512222+0000","last_unstale":"2023-10-06T11:41:56.519077+0000","last_undegraded":"2023-10-06T11:41:56.519077+0000","last_fullsized":"2023-10-06T11:41:56.519077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513731+0000","last_change":"2023-10-06T11:41:56.509931+0000","last_active":"2023-10-06T11:41:56.513731+0000","last_peered":"2023-10-06T11:41:56.513731+0000","last_clean":"2023-10-06T11:41:56.513731+0000","last_became_active":"2023-10-06T11:41:56.509393+0000","last_became_peered":"2023-10-06T11:41:56.509393+0000","last_unstale":"2023-10-06T11:41:56.513731+0000","last_undegraded":"2023-10-06T11:41:56.513731+0000","last_fullsized":"2023-10-06T11:41:56.513731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:56.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516425+0000","last_change":"2023-10-06T11:41:56.511208+0000","last_active":"2023-10-06T11:41:56.516425+0000","last_peered":"2023-10-06T11:41:56.516425+0000","last_clean":"2023-10-06T11:41:56.516425+0000","last_became_active":"2023-10-06T11:41:56.510934+0000","last_became_peered":"2023-10-06T11:41:56.510934+0000","last_unstale":"2023-10-06T11:41:56.516425+0000","last_undegraded":"2023-10-06T11:41:56.516425+0000","last_fullsized":"2023-10-06T11:41:56.516425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:21.656559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519759+0000","last_change":"2023-10-06T11:41:56.516474+0000","last_active":"2023-10-06T11:41:56.519759+0000","last_peered":"2023-10-06T11:41:56.519759+0000","last_clean":"2023-10-06T11:41:56.519759+0000","last_became_active":"2023-10-06T11:41:56.516168+0000","last_became_peered":"2023-10-06T11:41:56.516168+0000","last_unstale":"2023-10-06T11:41:56.519759+0000","last_undegraded":"2023-10-06T11:41:56.519759+0000","last_fullsized":"2023-10-06T11:41:56.519759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513822+0000","last_change":"2023-10-06T11:41:56.509889+0000","last_active":"2023-10-06T11:41:56.513822+0000","last_peered":"2023-10-06T11:41:56.513822+0000","last_clean":"2023-10-06T11:41:56.513822+0000","last_became_active":"2023-10-06T11:41:56.509181+0000","last_became_peered":"2023-10-06T11:41:56.509181+0000","last_unstale":"2023-10-06T11:41:56.513822+0000","last_undegraded":"2023-10-06T11:41:56.513822+0000","last_fullsized":"2023-10-06T11:41:56.513822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:42.296263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522246+0000","last_change":"2023-10-06T11:41:56.518850+0000","last_active":"2023-10-06T11:41:56.522246+0000","last_peered":"2023-10-06T11:41:56.522246+0000","last_clean":"2023-10-06T11:41:56.522246+0000","last_became_active":"2023-10-06T11:41:56.518578+0000","last_became_peered":"2023-10-06T11:41:56.518578+0000","last_unstale":"2023-10-06T11:41:56.522246+0000","last_undegraded":"2023-10-06T11:41:56.522246+0000","last_fullsized":"2023-10-06T11:41:56.522246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514003+0000","last_change":"2023-10-06T11:41:56.508470+0000","last_active":"2023-10-06T11:41:56.514003+0000","last_peered":"2023-10-06T11:41:56.514003+0000","last_clean":"2023-10-06T11:41:56.514003+0000","last_became_active":"2023-10-06T11:41:56.508167+0000","last_became_peered":"2023-10-06T11:41:56.508167+0000","last_unstale":"2023-10-06T11:41:56.514003+0000","last_undegraded":"2023-10-06T11:41:56.514003+0000","last_fullsized":"2023-10-06T11:41:56.514003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519630+0000","last_change":"2023-10-06T11:41:56.517434+0000","last_active":"2023-10-06T11:41:56.519630+0000","last_peered":"2023-10-06T11:41:56.519630+0000","last_clean":"2023-10-06T11:41:56.519630+0000","last_became_active":"2023-10-06T11:41:56.517143+0000","last_became_peered":"2023-10-06T11:41:56.517143+0000","last_unstale":"2023-10-06T11:41:56.519630+0000","last_undegraded":"2023-10-06T11:41:56.519630+0000","last_fullsized":"2023-10-06T11:41:56.519630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514049+0000","last_change":"2023-10-06T11:41:56.510953+0000","last_active":"2023-10-06T11:41:56.514049+0000","last_peered":"2023-10-06T11:41:56.514049+0000","last_clean":"2023-10-06T11:41:56.514049+0000","last_became_active":"2023-10-06T11:41:56.510684+0000","last_became_peered":"2023-10-06T11:41:56.510684+0000","last_unstale":"2023-10-06T11:41:56.514049+0000","last_undegraded":"2023-10-06T11:41:56.514049+0000","last_fullsized":"2023-10-06T11:41:56.514049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516780+0000","last_change":"2023-10-06T11:41:56.511114+0000","last_active":"2023-10-06T11:41:56.516780+0000","last_peered":"2023-10-06T11:41:56.516780+0000","last_clean":"2023-10-06T11:41:56.516780+0000","last_became_active":"2023-10-06T11:41:56.510653+0000","last_became_peered":"2023-10-06T11:41:56.510653+0000","last_unstale":"2023-10-06T11:41:56.516780+0000","last_undegraded":"2023-10-06T11:41:56.516780+0000","last_fullsized":"2023-10-06T11:41:56.516780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518118+0000","last_change":"2023-10-06T11:41:56.512980+0000","last_active":"2023-10-06T11:41:56.518118+0000","last_peered":"2023-10-06T11:41:56.518118+0000","last_clean":"2023-10-06T11:41:56.518118+0000","last_became_active":"2023-10-06T11:41:56.512699+0000","last_became_peered":"2023-10-06T11:41:56.512699+0000","last_unstale":"2023-10-06T11:41:56.518118+0000","last_undegraded":"2023-10-06T11:41:56.518118+0000","last_fullsized":"2023-10-06T11:41:56.518118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:20.998002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518013+0000","last_change":"2023-10-06T11:41:56.512383+0000","last_active":"2023-10-06T11:41:56.518013+0000","last_peered":"2023-10-06T11:41:56.518013+0000","last_clean":"2023-10-06T11:41:56.518013+0000","last_became_active":"2023-10-06T11:41:56.512086+0000","last_became_peered":"2023-10-06T11:41:56.512086+0000","last_unstale":"2023-10-06T11:41:56.518013+0000","last_undegraded":"2023-10-06T11:41:56.518013+0000","last_fullsized":"2023-10-06T11:41:56.518013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:44.564348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514263+0000","last_change":"2023-10-06T11:41:56.510312+0000","last_active":"2023-10-06T11:41:56.514263+0000","last_peered":"2023-10-06T11:41:56.514263+0000","last_clean":"2023-10-06T11:41:56.514263+0000","last_became_active":"2023-10-06T11:41:56.510046+0000","last_became_peered":"2023-10-06T11:41:56.510046+0000","last_unstale":"2023-10-06T11:41:56.514263+0000","last_undegraded":"2023-10-06T11:41:56.514263+0000","last_fullsized":"2023-10-06T11:41:56.514263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522268+0000","last_change":"2023-10-06T11:41:56.518929+0000","last_active":"2023-10-06T11:41:56.522268+0000","last_peered":"2023-10-06T11:41:56.522268+0000","last_clean":"2023-10-06T11:41:56.522268+0000","last_became_active":"2023-10-06T11:41:56.518671+0000","last_became_peered":"2023-10-06T11:41:56.518671+0000","last_unstale":"2023-10-06T11:41:56.522268+0000","last_undegraded":"2023-10-06T11:41:56.522268+0000","last_fullsized":"2023-10-06T11:41:56.522268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513758+0000","last_change":"2023-10-06T11:41:56.509452+0000","last_active":"2023-10-06T11:41:56.513758+0000","last_peered":"2023-10-06T11:41:56.513758+0000","last_clean":"2023-10-06T11:41:56.513758+0000","last_became_active":"2023-10-06T11:41:56.509141+0000","last_became_peered":"2023-10-06T11:41:56.509141+0000","last_unstale":"2023-10-06T11:41:56.513758+0000","last_undegraded":"2023-10-06T11:41:56.513758+0000","last_fullsized":"2023-10-06T11:41:56.513758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515869+0000","last_change":"2023-10-06T11:41:56.511167+0000","last_active":"2023-10-06T11:41:56.515869+0000","last_peered":"2023-10-06T11:41:56.515869+0000","last_clean":"2023-10-06T11:41:56.515869+0000","last_became_active":"2023-10-06T11:41:56.510849+0000","last_became_peered":"2023-10-06T11:41:56.510849+0000","last_unstale":"2023-10-06T11:41:56.515869+0000","last_undegraded":"2023-10-06T11:41:56.515869+0000","last_fullsized":"2023-10-06T11:41:56.515869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516793+0000","last_change":"2023-10-06T11:41:56.511248+0000","last_active":"2023-10-06T11:41:56.516793+0000","last_peered":"2023-10-06T11:41:56.516793+0000","last_clean":"2023-10-06T11:41:56.516793+0000","last_became_active":"2023-10-06T11:41:56.510023+0000","last_became_peered":"2023-10-06T11:41:56.510023+0000","last_unstale":"2023-10-06T11:41:56.516793+0000","last_undegraded":"2023-10-06T11:41:56.516793+0000","last_fullsized":"2023-10-06T11:41:56.516793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519577+0000","last_change":"2023-10-06T11:41:56.513818+0000","last_active":"2023-10-06T11:41:56.519577+0000","last_peered":"2023-10-06T11:41:56.519577+0000","last_clean":"2023-10-06T11:41:56.519577+0000","last_became_active":"2023-10-06T11:41:56.513508+0000","last_became_peered":"2023-10-06T11:41:56.513508+0000","last_unstale":"2023-10-06T11:41:56.519577+0000","last_undegraded":"2023-10-06T11:41:56.519577+0000","last_fullsized":"2023-10-06T11:41:56.519577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:25.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519973+0000","last_change":"2023-10-06T11:41:56.514028+0000","last_active":"2023-10-06T11:41:56.519973+0000","last_peered":"2023-10-06T11:41:56.519973+0000","last_clean":"2023-10-06T11:41:56.519973+0000","last_became_active":"2023-10-06T11:41:56.513176+0000","last_became_peered":"2023-10-06T11:41:56.513176+0000","last_unstale":"2023-10-06T11:41:56.519973+0000","last_undegraded":"2023-10-06T11:41:56.519973+0000","last_fullsized":"2023-10-06T11:41:56.519973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516670+0000","last_change":"2023-10-06T11:41:56.513150+0000","last_active":"2023-10-06T11:41:56.516670+0000","last_peered":"2023-10-06T11:41:56.516670+0000","last_clean":"2023-10-06T11:41:56.516670+0000","last_became_active":"2023-10-06T11:41:56.512844+0000","last_became_peered":"2023-10-06T11:41:56.512844+0000","last_unstale":"2023-10-06T11:41:56.516670+0000","last_undegraded":"2023-10-06T11:41:56.516670+0000","last_fullsized":"2023-10-06T11:41:56.516670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519663+0000","last_change":"2023-10-06T11:41:56.512179+0000","last_active":"2023-10-06T11:41:56.519663+0000","last_peered":"2023-10-06T11:41:56.519663+0000","last_clean":"2023-10-06T11:41:56.519663+0000","last_became_active":"2023-10-06T11:41:56.511885+0000","last_became_peered":"2023-10-06T11:41:56.511885+0000","last_unstale":"2023-10-06T11:41:56.519663+0000","last_undegraded":"2023-10-06T11:41:56.519663+0000","last_fullsized":"2023-10-06T11:41:56.519663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520009+0000","last_change":"2023-10-06T11:41:56.514447+0000","last_active":"2023-10-06T11:41:56.520009+0000","last_peered":"2023-10-06T11:41:56.520009+0000","last_clean":"2023-10-06T11:41:56.520009+0000","last_became_active":"2023-10-06T11:41:56.514134+0000","last_became_peered":"2023-10-06T11:41:56.514134+0000","last_unstale":"2023-10-06T11:41:56.520009+0000","last_undegraded":"2023-10-06T11:41:56.520009+0000","last_fullsized":"2023-10-06T11:41:56.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516608+0000","last_change":"2023-10-06T11:41:56.513062+0000","last_active":"2023-10-06T11:41:56.516608+0000","last_peered":"2023-10-06T11:41:56.516608+0000","last_clean":"2023-10-06T11:41:56.516608+0000","last_became_active":"2023-10-06T11:41:56.512812+0000","last_became_peered":"2023-10-06T11:41:56.512812+0000","last_unstale":"2023-10-06T11:41:56.516608+0000","last_undegraded":"2023-10-06T11:41:56.516608+0000","last_fullsized":"2023-10-06T11:41:56.516608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517749+0000","last_change":"2023-10-06T11:41:56.514788+0000","last_active":"2023-10-06T11:41:56.517749+0000","last_peered":"2023-10-06T11:41:56.517749+0000","last_clean":"2023-10-06T11:41:56.517749+0000","last_became_active":"2023-10-06T11:41:56.514494+0000","last_became_peered":"2023-10-06T11:41:56.514494+0000","last_unstale":"2023-10-06T11:41:56.517749+0000","last_undegraded":"2023-10-06T11:41:56.517749+0000","last_fullsized":"2023-10-06T11:41:56.517749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516257+0000","last_change":"2023-10-06T11:41:56.511137+0000","last_active":"2023-10-06T11:41:56.516257+0000","last_peered":"2023-10-06T11:41:56.516257+0000","last_clean":"2023-10-06T11:41:56.516257+0000","last_became_active":"2023-10-06T11:41:56.510826+0000","last_became_peered":"2023-10-06T11:41:56.510826+0000","last_unstale":"2023-10-06T11:41:56.516257+0000","last_undegraded":"2023-10-06T11:41:56.516257+0000","last_fullsized":"2023-10-06T11:41:56.516257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519152+0000","last_change":"2023-10-06T11:41:56.513345+0000","last_active":"2023-10-06T11:41:56.519152+0000","last_peered":"2023-10-06T11:41:56.519152+0000","last_clean":"2023-10-06T11:41:56.519152+0000","last_became_active":"2023-10-06T11:41:56.513068+0000","last_became_peered":"2023-10-06T11:41:56.513068+0000","last_unstale":"2023-10-06T11:41:56.519152+0000","last_undegraded":"2023-10-06T11:41:56.519152+0000","last_fullsized":"2023-10-06T11:41:56.519152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.529014+0000","last_change":"2023-10-06T11:41:55.504888+0000","last_active":"2023-10-06T11:41:56.529014+0000","last_peered":"2023-10-06T11:41:56.529014+0000","last_clean":"2023-10-06T11:41:56.529014+0000","last_became_active":"2023-10-06T11:41:55.504602+0000","last_became_peered":"2023-10-06T11:41:55.504602+0000","last_unstale":"2023-10-06T11:41:56.529014+0000","last_undegraded":"2023-10-06T11:41:56.529014+0000","last_fullsized":"2023-10-06T11:41:56.529014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518587+0000","last_change":"2023-10-06T11:41:56.512500+0000","last_active":"2023-10-06T11:41:56.518587+0000","last_peered":"2023-10-06T11:41:56.518587+0000","last_clean":"2023-10-06T11:41:56.518587+0000","last_became_active":"2023-10-06T11:41:56.512213+0000","last_became_peered":"2023-10-06T11:41:56.512213+0000","last_unstale":"2023-10-06T11:41:56.518587+0000","last_undegraded":"2023-10-06T11:41:56.518587+0000","last_fullsized":"2023-10-06T11:41:56.518587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497430+0000","last_change":"2023-10-06T11:41:55.505880+0000","last_active":"2023-10-06T11:41:56.497430+0000","last_peered":"2023-10-06T11:41:56.497430+0000","last_clean":"2023-10-06T11:41:56.497430+0000","last_became_active":"2023-10-06T11:41:55.504907+0000","last_became_peered":"2023-10-06T11:41:55.504907+0000","last_unstale":"2023-10-06T11:41:56.497430+0000","last_undegraded":"2023-10-06T11:41:56.497430+0000","last_fullsized":"2023-10-06T11:41:56.497430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519692+0000","last_change":"2023-10-06T11:41:56.514094+0000","last_active":"2023-10-06T11:41:56.519692+0000","last_peered":"2023-10-06T11:41:56.519692+0000","last_clean":"2023-10-06T11:41:56.519692+0000","last_became_active":"2023-10-06T11:41:56.513833+0000","last_became_peered":"2023-10-06T11:41:56.513833+0000","last_unstale":"2023-10-06T11:41:56.519692+0000","last_undegraded":"2023-10-06T11:41:56.519692+0000","last_fullsized":"2023-10-06T11:41:56.519692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503895+0000","last_change":"2023-10-06T11:41:55.504864+0000","last_active":"2023-10-06T11:41:56.503895+0000","last_peered":"2023-10-06T11:41:56.503895+0000","last_clean":"2023-10-06T11:41:56.503895+0000","last_became_active":"2023-10-06T11:41:55.504166+0000","last_became_peered":"2023-10-06T11:41:55.504166+0000","last_unstale":"2023-10-06T11:41:56.503895+0000","last_undegraded":"2023-10-06T11:41:56.503895+0000","last_fullsized":"2023-10-06T11:41:56.503895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514214+0000","last_change":"2023-10-06T11:41:56.509697+0000","last_active":"2023-10-06T11:41:56.514214+0000","last_peered":"2023-10-06T11:41:56.514214+0000","last_clean":"2023-10-06T11:41:56.514214+0000","last_became_active":"2023-10-06T11:41:56.509414+0000","last_became_peered":"2023-10-06T11:41:56.509414+0000","last_unstale":"2023-10-06T11:41:56.514214+0000","last_undegraded":"2023-10-06T11:41:56.514214+0000","last_fullsized":"2023-10-06T11:41:56.514214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507789+0000","last_change":"2023-10-06T11:41:55.496678+0000","last_active":"2023-10-06T11:41:56.507789+0000","last_peered":"2023-10-06T11:41:56.507789+0000","last_clean":"2023-10-06T11:41:56.507789+0000","last_became_active":"2023-10-06T11:41:55.494896+0000","last_became_peered":"2023-10-06T11:41:55.494896+0000","last_unstale":"2023-10-06T11:41:56.507789+0000","last_undegraded":"2023-10-06T11:41:56.507789+0000","last_fullsized":"2023-10-06T11:41:56.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517092+0000","last_change":"2023-10-06T11:41:56.512202+0000","last_active":"2023-10-06T11:41:56.517092+0000","last_peered":"2023-10-06T11:41:56.517092+0000","last_clean":"2023-10-06T11:41:56.517092+0000","last_became_active":"2023-10-06T11:41:56.511913+0000","last_became_peered":"2023-10-06T11:41:56.511913+0000","last_unstale":"2023-10-06T11:41:56.517092+0000","last_undegraded":"2023-10-06T11:41:56.517092+0000","last_fullsized":"2023-10-06T11:41:56.517092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502121+0000","last_change":"2023-10-06T11:41:55.500940+0000","last_active":"2023-10-06T11:41:56.502121+0000","last_peered":"2023-10-06T11:41:56.502121+0000","last_clean":"2023-10-06T11:41:56.502121+0000","last_became_active":"2023-10-06T11:41:55.500652+0000","last_became_peered":"2023-10-06T11:41:55.500652+0000","last_unstale":"2023-10-06T11:41:56.502121+0000","last_undegraded":"2023-10-06T11:41:56.502121+0000","last_fullsized":"2023-10-06T11:41:56.502121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520515+0000","last_change":"2023-10-06T11:41:56.515837+0000","last_active":"2023-10-06T11:41:56.520515+0000","last_peered":"2023-10-06T11:41:56.520515+0000","last_clean":"2023-10-06T11:41:56.520515+0000","last_became_active":"2023-10-06T11:41:56.515511+0000","last_became_peered":"2023-10-06T11:41:56.515511+0000","last_unstale":"2023-10-06T11:41:56.520515+0000","last_undegraded":"2023-10-06T11:41:56.520515+0000","last_fullsized":"2023-10-06T11:41:56.520515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534860+0000","last_change":"2023-10-06T11:41:55.509167+0000","last_active":"2023-10-06T11:41:56.534860+0000","last_peered":"2023-10-06T11:41:56.534860+0000","last_clean":"2023-10-06T11:41:56.534860+0000","last_became_active":"2023-10-06T11:41:55.508968+0000","last_became_peered":"2023-10-06T11:41:55.508968+0000","last_unstale":"2023-10-06T11:41:56.534860+0000","last_undegraded":"2023-10-06T11:41:56.534860+0000","last_fullsized":"2023-10-06T11:41:56.534860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:13.549315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520107+0000","last_change":"2023-10-06T11:41:56.515835+0000","last_active":"2023-10-06T11:41:56.520107+0000","last_peered":"2023-10-06T11:41:56.520107+0000","last_clean":"2023-10-06T11:41:56.520107+0000","last_became_active":"2023-10-06T11:41:56.514755+0000","last_became_peered":"2023-10-06T11:41:56.514755+0000","last_unstale":"2023-10-06T11:41:56.520107+0000","last_undegraded":"2023-10-06T11:41:56.520107+0000","last_fullsized":"2023-10-06T11:41:56.520107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.536294+0000","last_change":"2023-10-06T11:41:55.503423+0000","last_active":"2023-10-06T11:41:56.536294+0000","last_peered":"2023-10-06T11:41:56.536294+0000","last_clean":"2023-10-06T11:41:56.536294+0000","last_became_active":"2023-10-06T11:41:55.499943+0000","last_became_peered":"2023-10-06T11:41:55.499943+0000","last_unstale":"2023-10-06T11:41:56.536294+0000","last_undegraded":"2023-10-06T11:41:56.536294+0000","last_fullsized":"2023-10-06T11:41:56.536294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519939+0000","last_change":"2023-10-06T11:41:56.513385+0000","last_active":"2023-10-06T11:41:56.519939+0000","last_peered":"2023-10-06T11:41:56.519939+0000","last_clean":"2023-10-06T11:41:56.519939+0000","last_became_active":"2023-10-06T11:41:56.513091+0000","last_became_peered":"2023-10-06T11:41:56.513091+0000","last_unstale":"2023-10-06T11:41:56.519939+0000","last_undegraded":"2023-10-06T11:41:56.519939+0000","last_fullsized":"2023-10-06T11:41:56.519939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:31.044745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534779+0000","last_change":"2023-10-06T11:41:55.508640+0000","last_active":"2023-10-06T11:41:56.534779+0000","last_peered":"2023-10-06T11:41:56.534779+0000","last_clean":"2023-10-06T11:41:56.534779+0000","last_became_active":"2023-10-06T11:41:55.508359+0000","last_became_peered":"2023-10-06T11:41:55.508359+0000","last_unstale":"2023-10-06T11:41:56.534779+0000","last_undegraded":"2023-10-06T11:41:56.534779+0000","last_fullsized":"2023-10-06T11:41:56.534779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:20.652268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520221+0000","last_change":"2023-10-06T11:41:56.518196+0000","last_active":"2023-10-06T11:41:56.520221+0000","last_peered":"2023-10-06T11:41:56.520221+0000","last_clean":"2023-10-06T11:41:56.520221+0000","last_became_active":"2023-10-06T11:41:56.517922+0000","last_became_peered":"2023-10-06T11:41:56.517922+0000","last_unstale":"2023-10-06T11:41:56.520221+0000","last_undegraded":"2023-10-06T11:41:56.520221+0000","last_fullsized":"2023-10-06T11:41:56.520221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507442+0000","last_change":"2023-10-06T11:41:55.498881+0000","last_active":"2023-10-06T11:41:56.507442+0000","last_peered":"2023-10-06T11:41:56.507442+0000","last_clean":"2023-10-06T11:41:56.507442+0000","last_became_active":"2023-10-06T11:41:55.498646+0000","last_became_peered":"2023-10-06T11:41:55.498646+0000","last_unstale":"2023-10-06T11:41:56.507442+0000","last_undegraded":"2023-10-06T11:41:56.507442+0000","last_fullsized":"2023-10-06T11:41:56.507442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.521939+0000","last_change":"2023-10-06T11:41:56.514291+0000","last_active":"2023-10-06T11:41:56.521939+0000","last_peered":"2023-10-06T11:41:56.521939+0000","last_clean":"2023-10-06T11:41:56.521939+0000","last_became_active":"2023-10-06T11:41:56.513811+0000","last_became_peered":"2023-10-06T11:41:56.513811+0000","last_unstale":"2023-10-06T11:41:56.521939+0000","last_undegraded":"2023-10-06T11:41:56.521939+0000","last_fullsized":"2023-10-06T11:41:56.521939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506360+0000","last_change":"2023-10-06T11:41:55.510456+0000","last_active":"2023-10-06T11:41:56.506360+0000","last_peered":"2023-10-06T11:41:56.506360+0000","last_clean":"2023-10-06T11:41:56.506360+0000","last_became_active":"2023-10-06T11:41:55.510148+0000","last_became_peered":"2023-10-06T11:41:55.510148+0000","last_unstale":"2023-10-06T11:41:56.506360+0000","last_undegraded":"2023-10-06T11:41:56.506360+0000","last_fullsized":"2023-10-06T11:41:56.506360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519708+0000","last_change":"2023-10-06T11:41:56.516787+0000","last_active":"2023-10-06T11:41:56.519708+0000","last_peered":"2023-10-06T11:41:56.519708+0000","last_clean":"2023-10-06T11:41:56.519708+0000","last_became_active":"2023-10-06T11:41:56.516510+0000","last_became_peered":"2023-10-06T11:41:56.516510+0000","last_unstale":"2023-10-06T11:41:56.519708+0000","last_undegraded":"2023-10-06T11:41:56.519708+0000","last_fullsized":"2023-10-06T11:41:56.519708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.532331+0000","last_change":"2023-10-06T11:41:55.502794+0000","last_active":"2023-10-06T11:41:56.532331+0000","last_peered":"2023-10-06T11:41:56.532331+0000","last_clean":"2023-10-06T11:41:56.532331+0000","last_became_active":"2023-10-06T11:41:55.502522+0000","last_became_peered":"2023-10-06T11:41:55.502522+0000","last_unstale":"2023-10-06T11:41:56.532331+0000","last_undegraded":"2023-10-06T11:41:56.532331+0000","last_fullsized":"2023-10-06T11:41:56.532331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520360+0000","last_change":"2023-10-06T11:41:56.515679+0000","last_active":"2023-10-06T11:41:56.520360+0000","last_peered":"2023-10-06T11:41:56.520360+0000","last_clean":"2023-10-06T11:41:56.520360+0000","last_became_active":"2023-10-06T11:41:56.515167+0000","last_became_peered":"2023-10-06T11:41:56.515167+0000","last_unstale":"2023-10-06T11:41:56.520360+0000","last_undegraded":"2023-10-06T11:41:56.520360+0000","last_fullsized":"2023-10-06T11:41:56.520360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534434+0000","last_change":"2023-10-06T11:41:55.506331+0000","last_active":"2023-10-06T11:41:56.534434+0000","last_peered":"2023-10-06T11:41:56.534434+0000","last_clean":"2023-10-06T11:41:56.534434+0000","last_became_active":"2023-10-06T11:41:55.506044+0000","last_became_peered":"2023-10-06T11:41:55.506044+0000","last_unstale":"2023-10-06T11:41:56.534434+0000","last_undegraded":"2023-10-06T11:41:56.534434+0000","last_fullsized":"2023-10-06T11:41:56.534434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514023+0000","last_change":"2023-10-06T11:41:56.509291+0000","last_active":"2023-10-06T11:41:56.514023+0000","last_peered":"2023-10-06T11:41:56.514023+0000","last_clean":"2023-10-06T11:41:56.514023+0000","last_became_active":"2023-10-06T11:41:56.509004+0000","last_became_peered":"2023-10-06T11:41:56.509004+0000","last_unstale":"2023-10-06T11:41:56.514023+0000","last_undegraded":"2023-10-06T11:41:56.514023+0000","last_fullsized":"2023-10-06T11:41:56.514023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534956+0000","last_change":"2023-10-06T11:41:55.510357+0000","last_active":"2023-10-06T11:41:56.534956+0000","last_peered":"2023-10-06T11:41:56.534956+0000","last_clean":"2023-10-06T11:41:56.534956+0000","last_became_active":"2023-10-06T11:41:55.510053+0000","last_became_peered":"2023-10-06T11:41:55.510053+0000","last_unstale":"2023-10-06T11:41:56.534956+0000","last_undegraded":"2023-10-06T11:41:56.534956+0000","last_fullsized":"2023-10-06T11:41:56.534956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516748+0000","last_change":"2023-10-06T11:41:56.510920+0000","last_active":"2023-10-06T11:41:56.516748+0000","last_peered":"2023-10-06T11:41:56.516748+0000","last_clean":"2023-10-06T11:41:56.516748+0000","last_became_active":"2023-10-06T11:41:56.510620+0000","last_became_peered":"2023-10-06T11:41:56.510620+0000","last_unstale":"2023-10-06T11:41:56.516748+0000","last_undegraded":"2023-10-06T11:41:56.516748+0000","last_fullsized":"2023-10-06T11:41:56.516748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:01.202344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534773+0000","last_change":"2023-10-06T11:41:55.501172+0000","last_active":"2023-10-06T11:41:56.534773+0000","last_peered":"2023-10-06T11:41:56.534773+0000","last_clean":"2023-10-06T11:41:56.534773+0000","last_became_active":"2023-10-06T11:41:55.500891+0000","last_became_peered":"2023-10-06T11:41:55.500891+0000","last_unstale":"2023-10-06T11:41:56.534773+0000","last_undegraded":"2023-10-06T11:41:56.534773+0000","last_fullsized":"2023-10-06T11:41:56.534773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:31.984593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516634+0000","last_change":"2023-10-06T11:41:56.512054+0000","last_active":"2023-10-06T11:41:56.516634+0000","last_peered":"2023-10-06T11:41:56.516634+0000","last_clean":"2023-10-06T11:41:56.516634+0000","last_became_active":"2023-10-06T11:41:56.511385+0000","last_became_peered":"2023-10-06T11:41:56.511385+0000","last_unstale":"2023-10-06T11:41:56.516634+0000","last_undegraded":"2023-10-06T11:41:56.516634+0000","last_fullsized":"2023-10-06T11:41:56.516634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.394264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.501453+0000","last_change":"2023-10-06T11:41:55.505446+0000","last_active":"2023-10-06T11:41:56.501453+0000","last_peered":"2023-10-06T11:41:56.501453+0000","last_clean":"2023-10-06T11:41:56.501453+0000","last_became_active":"2023-10-06T11:41:55.505189+0000","last_became_peered":"2023-10-06T11:41:55.505189+0000","last_unstale":"2023-10-06T11:41:56.501453+0000","last_undegraded":"2023-10-06T11:41:56.501453+0000","last_fullsized":"2023-10-06T11:41:56.501453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519776+0000","last_change":"2023-10-06T11:41:56.517521+0000","last_active":"2023-10-06T11:41:56.519776+0000","last_peered":"2023-10-06T11:41:56.519776+0000","last_clean":"2023-10-06T11:41:56.519776+0000","last_became_active":"2023-10-06T11:41:56.517084+0000","last_became_peered":"2023-10-06T11:41:56.517084+0000","last_unstale":"2023-10-06T11:41:56.519776+0000","last_undegraded":"2023-10-06T11:41:56.519776+0000","last_fullsized":"2023-10-06T11:41:56.519776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531354+0000","last_change":"2023-10-06T11:41:55.509060+0000","last_active":"2023-10-06T11:41:56.531354+0000","last_peered":"2023-10-06T11:41:56.531354+0000","last_clean":"2023-10-06T11:41:56.531354+0000","last_became_active":"2023-10-06T11:41:55.508766+0000","last_became_peered":"2023-10-06T11:41:55.508766+0000","last_unstale":"2023-10-06T11:41:56.531354+0000","last_undegraded":"2023-10-06T11:41:56.531354+0000","last_fullsized":"2023-10-06T11:41:56.531354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514111+0000","last_change":"2023-10-06T11:41:56.510155+0000","last_active":"2023-10-06T11:41:56.514111+0000","last_peered":"2023-10-06T11:41:56.514111+0000","last_clean":"2023-10-06T11:41:56.514111+0000","last_became_active":"2023-10-06T11:41:56.509859+0000","last_became_peered":"2023-10-06T11:41:56.509859+0000","last_unstale":"2023-10-06T11:41:56.514111+0000","last_undegraded":"2023-10-06T11:41:56.514111+0000","last_fullsized":"2023-10-06T11:41:56.514111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507800+0000","last_change":"2023-10-06T11:41:55.511135+0000","last_active":"2023-10-06T11:41:56.507800+0000","last_peered":"2023-10-06T11:41:56.507800+0000","last_clean":"2023-10-06T11:41:56.507800+0000","last_became_active":"2023-10-06T11:41:55.510802+0000","last_became_peered":"2023-10-06T11:41:55.510802+0000","last_unstale":"2023-10-06T11:41:56.507800+0000","last_undegraded":"2023-10-06T11:41:56.507800+0000","last_fullsized":"2023-10-06T11:41:56.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516764+0000","last_change":"2023-10-06T11:41:56.510930+0000","last_active":"2023-10-06T11:41:56.516764+0000","last_peered":"2023-10-06T11:41:56.516764+0000","last_clean":"2023-10-06T11:41:56.516764+0000","last_became_active":"2023-10-06T11:41:56.510315+0000","last_became_peered":"2023-10-06T11:41:56.510315+0000","last_unstale":"2023-10-06T11:41:56.516764+0000","last_undegraded":"2023-10-06T11:41:56.516764+0000","last_fullsized":"2023-10-06T11:41:56.516764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500145+0000","last_change":"2023-10-06T11:41:55.506086+0000","last_active":"2023-10-06T11:41:56.500145+0000","last_peered":"2023-10-06T11:41:56.500145+0000","last_clean":"2023-10-06T11:41:56.500145+0000","last_became_active":"2023-10-06T11:41:55.505822+0000","last_became_peered":"2023-10-06T11:41:55.505822+0000","last_unstale":"2023-10-06T11:41:56.500145+0000","last_undegraded":"2023-10-06T11:41:56.500145+0000","last_fullsized":"2023-10-06T11:41:56.500145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520047+0000","last_change":"2023-10-06T11:41:56.514492+0000","last_active":"2023-10-06T11:41:56.520047+0000","last_peered":"2023-10-06T11:41:56.520047+0000","last_clean":"2023-10-06T11:41:56.520047+0000","last_became_active":"2023-10-06T11:41:56.513778+0000","last_became_peered":"2023-10-06T11:41:56.513778+0000","last_unstale":"2023-10-06T11:41:56.520047+0000","last_undegraded":"2023-10-06T11:41:56.520047+0000","last_fullsized":"2023-10-06T11:41:56.520047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503812+0000","last_change":"2023-10-06T11:41:55.505373+0000","last_active":"2023-10-06T11:41:56.503812+0000","last_peered":"2023-10-06T11:41:56.503812+0000","last_clean":"2023-10-06T11:41:56.503812+0000","last_became_active":"2023-10-06T11:41:55.505092+0000","last_became_peered":"2023-10-06T11:41:55.505092+0000","last_unstale":"2023-10-06T11:41:56.503812+0000","last_undegraded":"2023-10-06T11:41:56.503812+0000","last_fullsized":"2023-10-06T11:41:56.503812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516491+0000","last_change":"2023-10-06T11:41:56.512965+0000","last_active":"2023-10-06T11:41:56.516491+0000","last_peered":"2023-10-06T11:41:56.516491+0000","last_clean":"2023-10-06T11:41:56.516491+0000","last_became_active":"2023-10-06T11:41:56.512684+0000","last_became_peered":"2023-10-06T11:41:56.512684+0000","last_unstale":"2023-10-06T11:41:56.516491+0000","last_undegraded":"2023-10-06T11:41:56.516491+0000","last_fullsized":"2023-10-06T11:41:56.516491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534650+0000","last_change":"2023-10-06T11:41:55.510312+0000","last_active":"2023-10-06T11:41:56.534650+0000","last_peered":"2023-10-06T11:41:56.534650+0000","last_clean":"2023-10-06T11:41:56.534650+0000","last_became_active":"2023-10-06T11:41:55.510044+0000","last_became_peered":"2023-10-06T11:41:55.510044+0000","last_unstale":"2023-10-06T11:41:56.534650+0000","last_undegraded":"2023-10-06T11:41:56.534650+0000","last_fullsized":"2023-10-06T11:41:56.534650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500088+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:41:56.500088+0000","last_peered":"2023-10-06T11:41:56.500088+0000","last_clean":"2023-10-06T11:41:56.500088+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:41:56.500088+0000","last_undegraded":"2023-10-06T11:41:56.500088+0000","last_fullsized":"2023-10-06T11:41:56.500088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519625+0000","last_change":"2023-10-06T11:41:56.515565+0000","last_active":"2023-10-06T11:41:56.519625+0000","last_peered":"2023-10-06T11:41:56.519625+0000","last_clean":"2023-10-06T11:41:56.519625+0000","last_became_active":"2023-10-06T11:41:56.515007+0000","last_became_peered":"2023-10-06T11:41:56.515007+0000","last_unstale":"2023-10-06T11:41:56.519625+0000","last_undegraded":"2023-10-06T11:41:56.519625+0000","last_fullsized":"2023-10-06T11:41:56.519625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503231+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:41:56.503231+0000","last_peered":"2023-10-06T11:41:56.503231+0000","last_clean":"2023-10-06T11:41:56.503231+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:41:56.503231+0000","last_undegraded":"2023-10-06T11:41:56.503231+0000","last_fullsized":"2023-10-06T11:41:56.503231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507965+0000","last_change":"2023-10-06T11:41:55.500074+0000","last_active":"2023-10-06T11:41:56.507965+0000","last_peered":"2023-10-06T11:41:56.507965+0000","last_clean":"2023-10-06T11:41:56.507965+0000","last_became_active":"2023-10-06T11:41:55.499784+0000","last_became_peered":"2023-10-06T11:41:55.499784+0000","last_unstale":"2023-10-06T11:41:56.507965+0000","last_undegraded":"2023-10-06T11:41:56.507965+0000","last_fullsized":"2023-10-06T11:41:56.507965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513641+0000","last_change":"2023-10-06T11:41:56.509030+0000","last_active":"2023-10-06T11:41:56.513641+0000","last_peered":"2023-10-06T11:41:56.513641+0000","last_clean":"2023-10-06T11:41:56.513641+0000","last_became_active":"2023-10-06T11:41:56.508720+0000","last_became_peered":"2023-10-06T11:41:56.508720+0000","last_unstale":"2023-10-06T11:41:56.513641+0000","last_undegraded":"2023-10-06T11:41:56.513641+0000","last_fullsized":"2023-10-06T11:41:56.513641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:51.155604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497934+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:41:56.497934+0000","last_peered":"2023-10-06T11:41:56.497934+0000","last_clean":"2023-10-06T11:41:56.497934+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:41:56.497934+0000","last_undegraded":"2023-10-06T11:41:56.497934+0000","last_fullsized":"2023-10-06T11:41:56.497934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506182+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:41:56.506182+0000","last_peered":"2023-10-06T11:41:56.506182+0000","last_clean":"2023-10-06T11:41:56.506182+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:41:56.506182+0000","last_undegraded":"2023-10-06T11:41:56.506182+0000","last_fullsized":"2023-10-06T11:41:56.506182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507640+0000","last_change":"2023-10-06T11:41:55.506608+0000","last_active":"2023-10-06T11:41:56.507640+0000","last_peered":"2023-10-06T11:41:56.507640+0000","last_clean":"2023-10-06T11:41:56.507640+0000","last_became_active":"2023-10-06T11:41:55.506234+0000","last_became_peered":"2023-10-06T11:41:55.506234+0000","last_unstale":"2023-10-06T11:41:56.507640+0000","last_undegraded":"2023-10-06T11:41:56.507640+0000","last_fullsized":"2023-10-06T11:41:56.507640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519724+0000","last_change":"2023-10-06T11:41:56.516915+0000","last_active":"2023-10-06T11:41:56.519724+0000","last_peered":"2023-10-06T11:41:56.519724+0000","last_clean":"2023-10-06T11:41:56.519724+0000","last_became_active":"2023-10-06T11:41:56.516184+0000","last_became_peered":"2023-10-06T11:41:56.516184+0000","last_unstale":"2023-10-06T11:41:56.519724+0000","last_undegraded":"2023-10-06T11:41:56.519724+0000","last_fullsized":"2023-10-06T11:41:56.519724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505779+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:41:56.505779+0000","last_peered":"2023-10-06T11:41:56.505779+0000","last_clean":"2023-10-06T11:41:56.505779+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:41:56.505779+0000","last_undegraded":"2023-10-06T11:41:56.505779+0000","last_fullsized":"2023-10-06T11:41:56.505779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497472+0000","last_change":"2023-10-06T11:41:55.506060+0000","last_active":"2023-10-06T11:41:56.497472+0000","last_peered":"2023-10-06T11:41:56.497472+0000","last_clean":"2023-10-06T11:41:56.497472+0000","last_became_active":"2023-10-06T11:41:55.505551+0000","last_became_peered":"2023-10-06T11:41:55.505551+0000","last_unstale":"2023-10-06T11:41:56.497472+0000","last_undegraded":"2023-10-06T11:41:56.497472+0000","last_fullsized":"2023-10-06T11:41:56.497472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:23.340598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515790+0000","last_change":"2023-10-06T11:41:56.509141+0000","last_active":"2023-10-06T11:41:56.515790+0000","last_peered":"2023-10-06T11:41:56.515790+0000","last_clean":"2023-10-06T11:41:56.515790+0000","last_became_active":"2023-10-06T11:41:56.508814+0000","last_became_peered":"2023-10-06T11:41:56.508814+0000","last_unstale":"2023-10-06T11:41:56.515790+0000","last_undegraded":"2023-10-06T11:41:56.515790+0000","last_fullsized":"2023-10-06T11:41:56.515790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505943+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:41:56.505943+0000","last_peered":"2023-10-06T11:41:56.505943+0000","last_clean":"2023-10-06T11:41:56.505943+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:41:56.505943+0000","last_undegraded":"2023-10-06T11:41:56.505943+0000","last_fullsized":"2023-10-06T11:41:56.505943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502767+0000","last_change":"2023-10-06T11:41:55.504016+0000","last_active":"2023-10-06T11:41:56.502767+0000","last_peered":"2023-10-06T11:41:56.502767+0000","last_clean":"2023-10-06T11:41:56.502767+0000","last_became_active":"2023-10-06T11:41:55.503831+0000","last_became_peered":"2023-10-06T11:41:55.503831+0000","last_unstale":"2023-10-06T11:41:56.502767+0000","last_undegraded":"2023-10-06T11:41:56.502767+0000","last_fullsized":"2023-10-06T11:41:56.502767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519487+0000","last_change":"2023-10-06T11:41:56.515759+0000","last_active":"2023-10-06T11:41:56.519487+0000","last_peered":"2023-10-06T11:41:56.519487+0000","last_clean":"2023-10-06T11:41:56.519487+0000","last_became_active":"2023-10-06T11:41:56.514780+0000","last_became_peered":"2023-10-06T11:41:56.514780+0000","last_unstale":"2023-10-06T11:41:56.519487+0000","last_undegraded":"2023-10-06T11:41:56.519487+0000","last_fullsized":"2023-10-06T11:41:56.519487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502819+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:41:56.502819+0000","last_peered":"2023-10-06T11:41:56.502819+0000","last_clean":"2023-10-06T11:41:56.502819+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:41:56.502819+0000","last_undegraded":"2023-10-06T11:41:56.502819+0000","last_fullsized":"2023-10-06T11:41:56.502819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500296+0000","last_change":"2023-10-06T11:41:55.508163+0000","last_active":"2023-10-06T11:41:56.500296+0000","last_peered":"2023-10-06T11:41:56.500296+0000","last_clean":"2023-10-06T11:41:56.500296+0000","last_became_active":"2023-10-06T11:41:55.507855+0000","last_became_peered":"2023-10-06T11:41:55.507855+0000","last_unstale":"2023-10-06T11:41:56.500296+0000","last_undegraded":"2023-10-06T11:41:56.500296+0000","last_fullsized":"2023-10-06T11:41:56.500296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516585+0000","last_change":"2023-10-06T11:41:56.512837+0000","last_active":"2023-10-06T11:41:56.516585+0000","last_peered":"2023-10-06T11:41:56.516585+0000","last_clean":"2023-10-06T11:41:56.516585+0000","last_became_active":"2023-10-06T11:41:56.511727+0000","last_became_peered":"2023-10-06T11:41:56.511727+0000","last_unstale":"2023-10-06T11:41:56.516585+0000","last_undegraded":"2023-10-06T11:41:56.516585+0000","last_fullsized":"2023-10-06T11:41:56.516585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507674+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:41:56.507674+0000","last_peered":"2023-10-06T11:41:56.507674+0000","last_clean":"2023-10-06T11:41:56.507674+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:41:56.507674+0000","last_undegraded":"2023-10-06T11:41:56.507674+0000","last_fullsized":"2023-10-06T11:41:56.507674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497703+0000","last_change":"2023-10-06T11:41:55.508524+0000","last_active":"2023-10-06T11:41:56.497703+0000","last_peered":"2023-10-06T11:41:56.497703+0000","last_clean":"2023-10-06T11:41:56.497703+0000","last_became_active":"2023-10-06T11:41:55.505590+0000","last_became_peered":"2023-10-06T11:41:55.505590+0000","last_unstale":"2023-10-06T11:41:56.497703+0000","last_undegraded":"2023-10-06T11:41:56.497703+0000","last_fullsized":"2023-10-06T11:41:56.497703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.511544+0000","last_change":"2023-10-06T11:41:56.509540+0000","last_active":"2023-10-06T11:41:56.511544+0000","last_peered":"2023-10-06T11:41:56.511544+0000","last_clean":"2023-10-06T11:41:56.511544+0000","last_became_active":"2023-10-06T11:41:56.509208+0000","last_became_peered":"2023-10-06T11:41:56.509208+0000","last_unstale":"2023-10-06T11:41:56.511544+0000","last_undegraded":"2023-10-06T11:41:56.511544+0000","last_fullsized":"2023-10-06T11:41:56.511544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507693+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:41:56.507693+0000","last_peered":"2023-10-06T11:41:56.507693+0000","last_clean":"2023-10-06T11:41:56.507693+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:41:56.507693+0000","last_undegraded":"2023-10-06T11:41:56.507693+0000","last_fullsized":"2023-10-06T11:41:56.507693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503179+0000","last_change":"2023-10-06T11:41:55.505392+0000","last_active":"2023-10-06T11:41:56.503179+0000","last_peered":"2023-10-06T11:41:56.503179+0000","last_clean":"2023-10-06T11:41:56.503179+0000","last_became_active":"2023-10-06T11:41:55.505103+0000","last_became_peered":"2023-10-06T11:41:55.505103+0000","last_unstale":"2023-10-06T11:41:56.503179+0000","last_undegraded":"2023-10-06T11:41:56.503179+0000","last_fullsized":"2023-10-06T11:41:56.503179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516182+0000","last_change":"2023-10-06T11:41:56.509787+0000","last_active":"2023-10-06T11:41:56.516182+0000","last_peered":"2023-10-06T11:41:56.516182+0000","last_clean":"2023-10-06T11:41:56.516182+0000","last_became_active":"2023-10-06T11:41:56.509495+0000","last_became_peered":"2023-10-06T11:41:56.509495+0000","last_unstale":"2023-10-06T11:41:56.516182+0000","last_undegraded":"2023-10-06T11:41:56.516182+0000","last_fullsized":"2023-10-06T11:41:56.516182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531881+0000","last_change":"2023-10-06T11:41:55.501475+0000","last_active":"2023-10-06T11:41:56.531881+0000","last_peered":"2023-10-06T11:41:56.531881+0000","last_clean":"2023-10-06T11:41:56.531881+0000","last_became_active":"2023-10-06T11:41:55.500531+0000","last_became_peered":"2023-10-06T11:41:55.500531+0000","last_unstale":"2023-10-06T11:41:56.531881+0000","last_undegraded":"2023-10-06T11:41:56.531881+0000","last_fullsized":"2023-10-06T11:41:56.531881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519814+0000","last_change":"2023-10-06T11:41:56.518985+0000","last_active":"2023-10-06T11:41:56.519814+0000","last_peered":"2023-10-06T11:41:56.519814+0000","last_clean":"2023-10-06T11:41:56.519814+0000","last_became_active":"2023-10-06T11:41:56.518656+0000","last_became_peered":"2023-10-06T11:41:56.518656+0000","last_unstale":"2023-10-06T11:41:56.519814+0000","last_undegraded":"2023-10-06T11:41:56.519814+0000","last_fullsized":"2023-10-06T11:41:56.519814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:21.108193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534877+0000","last_change":"2023-10-06T11:41:55.506318+0000","last_active":"2023-10-06T11:41:56.534877+0000","last_peered":"2023-10-06T11:41:56.534877+0000","last_clean":"2023-10-06T11:41:56.534877+0000","last_became_active":"2023-10-06T11:41:55.506045+0000","last_became_peered":"2023-10-06T11:41:55.506045+0000","last_unstale":"2023-10-06T11:41:56.534877+0000","last_undegraded":"2023-10-06T11:41:56.534877+0000","last_fullsized":"2023-10-06T11:41:56.534877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520066+0000","last_change":"2023-10-06T11:41:56.515883+0000","last_active":"2023-10-06T11:41:56.520066+0000","last_peered":"2023-10-06T11:41:56.520066+0000","last_clean":"2023-10-06T11:41:56.520066+0000","last_became_active":"2023-10-06T11:41:56.515622+0000","last_became_peered":"2023-10-06T11:41:56.515622+0000","last_unstale":"2023-10-06T11:41:56.520066+0000","last_undegraded":"2023-10-06T11:41:56.520066+0000","last_fullsized":"2023-10-06T11:41:56.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:31.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.535054+0000","last_change":"2023-10-06T11:41:55.507981+0000","last_active":"2023-10-06T11:41:56.535054+0000","last_peered":"2023-10-06T11:41:56.535054+0000","last_clean":"2023-10-06T11:41:56.535054+0000","last_became_active":"2023-10-06T11:41:55.507665+0000","last_became_peered":"2023-10-06T11:41:55.507665+0000","last_unstale":"2023-10-06T11:41:56.535054+0000","last_undegraded":"2023-10-06T11:41:56.535054+0000","last_fullsized":"2023-10-06T11:41:56.535054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513856+0000","last_change":"2023-10-06T11:41:56.510088+0000","last_active":"2023-10-06T11:41:56.513856+0000","last_peered":"2023-10-06T11:41:56.513856+0000","last_clean":"2023-10-06T11:41:56.513856+0000","last_became_active":"2023-10-06T11:41:56.509600+0000","last_became_peered":"2023-10-06T11:41:56.509600+0000","last_unstale":"2023-10-06T11:41:56.513856+0000","last_undegraded":"2023-10-06T11:41:56.513856+0000","last_fullsized":"2023-10-06T11:41:56.513856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:16.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534276+0000","last_change":"2023-10-06T11:41:55.503624+0000","last_active":"2023-10-06T11:41:56.534276+0000","last_peered":"2023-10-06T11:41:56.534276+0000","last_clean":"2023-10-06T11:41:56.534276+0000","last_became_active":"2023-10-06T11:41:55.503318+0000","last_became_peered":"2023-10-06T11:41:55.503318+0000","last_unstale":"2023-10-06T11:41:56.534276+0000","last_undegraded":"2023-10-06T11:41:56.534276+0000","last_fullsized":"2023-10-06T11:41:56.534276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:44.190608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519991+0000","last_change":"2023-10-06T11:41:56.510835+0000","last_active":"2023-10-06T11:41:56.519991+0000","last_peered":"2023-10-06T11:41:56.519991+0000","last_clean":"2023-10-06T11:41:56.519991+0000","last_became_active":"2023-10-06T11:41:56.510547+0000","last_became_peered":"2023-10-06T11:41:56.510547+0000","last_unstale":"2023-10-06T11:41:56.519991+0000","last_undegraded":"2023-10-06T11:41:56.519991+0000","last_fullsized":"2023-10-06T11:41:56.519991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534863+0000","last_change":"2023-10-06T11:41:55.497178+0000","last_active":"2023-10-06T11:41:56.534863+0000","last_peered":"2023-10-06T11:41:56.534863+0000","last_clean":"2023-10-06T11:41:56.534863+0000","last_became_active":"2023-10-06T11:41:55.495336+0000","last_became_peered":"2023-10-06T11:41:55.495336+0000","last_unstale":"2023-10-06T11:41:56.534863+0000","last_undegraded":"2023-10-06T11:41:56.534863+0000","last_fullsized":"2023-10-06T11:41:56.534863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519888+0000","last_change":"2023-10-06T11:41:56.511788+0000","last_active":"2023-10-06T11:41:56.519888+0000","last_peered":"2023-10-06T11:41:56.519888+0000","last_clean":"2023-10-06T11:41:56.519888+0000","last_became_active":"2023-10-06T11:41:56.511488+0000","last_became_peered":"2023-10-06T11:41:56.511488+0000","last_unstale":"2023-10-06T11:41:56.519888+0000","last_undegraded":"2023-10-06T11:41:56.519888+0000","last_fullsized":"2023-10-06T11:41:56.519888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516639+0000","last_change":"2023-10-06T11:41:56.511919+0000","last_active":"2023-10-06T11:41:56.516639+0000","last_peered":"2023-10-06T11:41:56.516639+0000","last_clean":"2023-10-06T11:41:56.516639+0000","last_became_active":"2023-10-06T11:41:56.511644+0000","last_became_peered":"2023-10-06T11:41:56.511644+0000","last_unstale":"2023-10-06T11:41:56.516639+0000","last_undegraded":"2023-10-06T11:41:56.516639+0000","last_fullsized":"2023-10-06T11:41:56.516639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520154+0000","last_change":"2023-10-06T11:41:56.516583+0000","last_active":"2023-10-06T11:41:56.520154+0000","last_peered":"2023-10-06T11:41:56.520154+0000","last_clean":"2023-10-06T11:41:56.520154+0000","last_became_active":"2023-10-06T11:41:56.516241+0000","last_became_peered":"2023-10-06T11:41:56.516241+0000","last_unstale":"2023-10-06T11:41:56.520154+0000","last_undegraded":"2023-10-06T11:41:56.520154+0000","last_fullsized":"2023-10-06T11:41:56.520154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520023+0000","last_change":"2023-10-06T11:41:56.514199+0000","last_active":"2023-10-06T11:41:56.520023+0000","last_peered":"2023-10-06T11:41:56.520023+0000","last_clean":"2023-10-06T11:41:56.520023+0000","last_became_active":"2023-10-06T11:41:56.513961+0000","last_became_peered":"2023-10-06T11:41:56.513961+0000","last_unstale":"2023-10-06T11:41:56.520023+0000","last_undegraded":"2023-10-06T11:41:56.520023+0000","last_fullsized":"2023-10-06T11:41:56.520023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520267+0000","last_change":"2023-10-06T11:41:56.518788+0000","last_active":"2023-10-06T11:41:56.520267+0000","last_peered":"2023-10-06T11:41:56.520267+0000","last_clean":"2023-10-06T11:41:56.520267+0000","last_became_active":"2023-10-06T11:41:56.516640+0000","last_became_peered":"2023-10-06T11:41:56.516640+0000","last_unstale":"2023-10-06T11:41:56.520267+0000","last_undegraded":"2023-10-06T11:41:56.520267+0000","last_fullsized":"2023-10-06T11:41:56.520267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520067+0000","last_change":"2023-10-06T11:41:56.516139+0000","last_active":"2023-10-06T11:41:56.520067+0000","last_peered":"2023-10-06T11:41:56.520067+0000","last_clean":"2023-10-06T11:41:56.520067+0000","last_became_active":"2023-10-06T11:41:56.514921+0000","last_became_peered":"2023-10-06T11:41:56.514921+0000","last_unstale":"2023-10-06T11:41:56.520067+0000","last_undegraded":"2023-10-06T11:41:56.520067+0000","last_fullsized":"2023-10-06T11:41:56.520067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":671324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":72139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":71675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:01.055 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:42:01.055 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T11:42:01.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T11:42:01.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T11:42:01.056 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T11:42:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.206+0000 7fa25fc48700 1 Processor -- start 2023-10-06T11:42:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.206+0000 7fa25fc48700 1 -- start start 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa258156550 0x7fa258069270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa2580b0ea0 0x7fa258156920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa25806ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580b2210 con 0x7fa258156550 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2580b1600 con 0x7fa2580b0ea0 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25fc48700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580b18e0 con 0x7fa258156e60 2023-10-06T11:42:01.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25d9e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa258156550 0x7fa258069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25d9e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa258156550 0x7fa258069270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42988/0 (socket says 172.21.15.16:42988) 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.207+0000 7fa25d9e4700 1 -- 172.21.15.16:0/1827110559 learned_addr learned my addr 172.21.15.16:0/1827110559 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d1e3700 1 --2- 172.21.15.16:0/1827110559 >> v2:172.21.15.177:3300/0 conn(0x7fa2580b0ea0 0x7fa258156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25e1e5700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa25806ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 -- 172.21.15.16:0/1827110559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 msgr2=0x7fa25806ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa25806ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 -- 172.21.15.16:0/1827110559 >> v2:172.21.15.177:3300/0 conn(0x7fa2580b0ea0 msgr2=0x7fa258156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/1827110559 >> v2:172.21.15.177:3300/0 conn(0x7fa2580b0ea0 0x7fa258156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 -- 172.21.15.16:0/1827110559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580b1bc0 con 0x7fa258156550 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa258156550 0x7fa258069270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa25806bd20 tx=0x7fa24800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14457eff6c13cacd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25c9e2700 1 -- 172.21.15.16:0/1827110559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248010070 con 0x7fa258156550 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25c9e2700 1 -- 172.21.15.16:0/1827110559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24800dd30 con 0x7fa258156550 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.208+0000 7fa25c9e2700 1 -- 172.21.15.16:0/1827110559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800dea0 con 0x7fa258156550 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 -- 172.21.15.16:0/1827110559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa258156550 msgr2=0x7fa258069270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa258156550 0x7fa258069270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa25806bd20 tx=0x7fa24800ba50 comp rx=0 tx=0).stop 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 -- 172.21.15.16:0/1827110559 shutdown_connections 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa25806ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 --2- 172.21.15.16:0/1827110559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa258156550 0x7fa258069270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 --2- 172.21.15.16:0/1827110559 >> v2:172.21.15.177:3300/0 conn(0x7fa2580b0ea0 0x7fa258156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 -- 172.21.15.16:0/1827110559 >> 172.21.15.16:0/1827110559 conn(0x7fa2580cd480 msgr2=0x7fa2580cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 -- 172.21.15.16:0/1827110559 shutdown_connections 2023-10-06T11:42:01.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.209+0000 7fa25fc48700 1 -- 172.21.15.16:0/1827110559 wait complete. 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.210+0000 7fa25fc48700 1 Processor -- start 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.210+0000 7fa25fc48700 1 -- start start 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.210+0000 7fa25fc48700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.210+0000 7fa25fc48700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa258156550 0x7fa2580c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25fc48700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa258151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25fc48700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580c6740 con 0x7fa2580b0ea0 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25fc48700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580c6a20 con 0x7fa258156550 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25fc48700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580c6d00 con 0x7fa258156e60 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43000/0 (socket says 172.21.15.16:43000) 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 -- 172.21.15.16:0/183632056 learned_addr learned my addr 172.21.15.16:0/183632056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d1e3700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa258156550 0x7fa2580c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25e1e5700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa258151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 -- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 msgr2=0x7fa258151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa258151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 -- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa258156550 msgr2=0x7fa2580c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa258156550 0x7fa2580c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 -- 172.21.15.16:0/183632056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24800bcf0 con 0x7fa2580b0ea0 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25e1e5700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa258151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25d9e4700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa24800d760 tx=0x7fa24801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800a980 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24800aae0 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa258153a40 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.211+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24800ae60 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.212+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581536e0 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.212+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa258153f60 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.213+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa248003040 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.213+0000 7fa255ffb700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 0x7fa2440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.213+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa248085a60 con 0x7fa2580b0ea0 2023-10-06T11:42:01.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.213+0000 7fa25d1e3700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 0x7fa2440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.214+0000 7fa25d1e3700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 0x7fa2440693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2580c9050 tx=0x7fa24c0091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.217+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa258153f60 con 0x7fa2580b0ea0 2023-10-06T11:42:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.465+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa258153f60 con 0x7fa2580b0ea0 2023-10-06T11:42:01.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.468+0000 7fa255ffb700 1 -- 172.21.15.16:0/183632056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163706 (secure 0 0 0) 0x7fa248052ad0 con 0x7fa2580b0ea0 2023-10-06T11:42:01.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:01.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 msgr2=0x7fa2440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 0x7fa2440693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa2580c9050 tx=0x7fa24c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 msgr2=0x7fa2581531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa24800d760 tx=0x7fa24801a000 comp rx=0 tx=0).stop 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 shutdown_connections 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2580b0ea0 0x7fa2581531a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa244066f30 0x7fa2440693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa258156e60 0x7fa258151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 --2- 172.21.15.16:0/183632056 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa258156550 0x7fa2580c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 >> 172.21.15.16:0/183632056 conn(0x7fa2580cd480 msgr2=0x7fa2580cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 shutdown_connections 2023-10-06T11:42:01.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.471+0000 7fa25fc48700 1 -- 172.21.15.16:0/183632056 wait complete. 2023-10-06T11:42:01.485 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":90263},{"type":"v1","addr":"172.21.15.177:6833","nonce":90263}]},"active_addr":"172.21.15.177:6833/90263","active_change":"2023-10-06T11:41:44.052151+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4115,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":3872668270}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2744304064}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":1329914895}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.177:0","nonce":2310367126}]}]} 2023-10-06T11:42:01.488 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T11:42:01.489 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T11:42:01.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:01.568 INFO:tasks.ceph.mds.b.smithi177.stderr:2023-10-06T11:42:01.567+0000 7f65a3004700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:42:01.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.606+0000 7f647f0ed700 1 Processor -- start 2023-10-06T11:42:01.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.606+0000 7f647f0ed700 1 -- start start 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f647f0ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64780cd870 0x7f64780ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f647f0ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64780cf280 0x7f64780c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f647f0ed700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64780ccb20 con 0x7f647815cc20 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f647f0ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64780b3110 con 0x7f64780cd870 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f647f0ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64780cbf10 con 0x7f64780cf280 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f6477fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64780cd870 0x7f64780ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f6477fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64780cd870 0x7f64780ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43002/0 (socket says 172.21.15.16:43002) 2023-10-06T11:42:01.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.607+0000 7f6477fff700 1 -- 172.21.15.16:0/2310963527 learned_addr learned my addr 172.21.15.16:0/2310963527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647d68a700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64780cf280 0x7f64780c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f6477fff700 1 -- 172.21.15.16:0/2310963527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64780cf280 msgr2=0x7f64780c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f6477fff700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64780cf280 0x7f64780c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f6477fff700 1 -- 172.21.15.16:0/2310963527 >> v1:172.21.15.177:6789/0 conn(0x7f647815cc20 legacy=0x7f64780cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f6477fff700 1 -- 172.21.15.16:0/2310963527 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64780cc1f0 con 0x7f64780cd870 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f6477fff700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cd870 0x7f64780ced40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f646c0099b0 tx=0x7f646c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e607cdca7c93da20 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f64777fe700 1 -- 172.21.15.16:0/2310963527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c00b820 con 0x7f64780cd870 2023-10-06T11:42:01.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f64777fe700 1 -- 172.21.15.16:0/2310963527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c00b980 con 0x7f64780cd870 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f64777fe700 1 -- 172.21.15.16:0/2310963527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c00bd40 con 0x7f64780cd870 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 -- 172.21.15.16:0/2310963527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cd870 msgr2=0x7f64780ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cd870 0x7f64780ced40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f646c0099b0 tx=0x7f646c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 -- 172.21.15.16:0/2310963527 shutdown_connections 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64780cf280 0x7f64780c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2310963527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cd870 0x7f64780ced40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 -- 172.21.15.16:0/2310963527 >> 172.21.15.16:0/2310963527 conn(0x7f6478099b40 msgr2=0x7f647806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.608+0000 7f647f0ed700 1 -- 172.21.15.16:0/2310963527 shutdown_connections 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.609+0000 7f647f0ed700 1 -- 172.21.15.16:0/2310963527 wait complete. 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 Processor -- start 2023-10-06T11:42:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 -- start start 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 0x7f64780c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f647815cc20 0x7f64781536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64780ca110 con 0x7f64780cf280 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64780b53d0 con 0x7f64780cd870 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.610+0000 7f647f0ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64780c9500 con 0x7f647815cc20 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43010/0 (socket says 172.21.15.16:43010) 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 -- 172.21.15.16:0/2957909797 learned_addr learned my addr 172.21.15.16:0/2957909797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 -- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f647815cc20 msgr2=0x7f64781536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f647d68a700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f647815cc20 0x7f64781536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f647ce89700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 0x7f64780c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f647815cc20 0x7f64781536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 -- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 msgr2=0x7f64780c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 0x7f64780c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 -- 172.21.15.16:0/2957909797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0094f0 con 0x7f64780cf280 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f647ce89700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 0x7f64780c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6477fff700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6478089390 tx=0x7f646c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c011c80 con 0x7f64780cf280 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c020070 con 0x7f64780cf280 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64780c97e0 con 0x7f64780cf280 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c01b6a0 con 0x7f64780cf280 2023-10-06T11:42:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.611+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64780c9da0 con 0x7f64780cf280 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.612+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647804c380 con 0x7f64780cf280 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.612+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f646c022030 con 0x7f64780cf280 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.613+0000 7f6474ff9700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 0x7f64640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.613+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f646c086510 con 0x7f64780cf280 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.613+0000 7f647ce89700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 0x7f64640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.614+0000 7f647ce89700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 0x7f64640693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6468002880 tx=0x7f6468009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.617+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f646c053620 con 0x7f64780cf280 2023-10-06T11:42:01.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.810+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f647804c380 con 0x7f64780cf280 2023-10-06T11:42:01.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.811+0000 7f6474ff9700 1 -- 172.21.15.16:0/2957909797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f646c056c40 con 0x7f64780cf280 2023-10-06T11:42:01.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:01.814 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 msgr2=0x7f64640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 0x7f64640693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6468002880 tx=0x7f6468009040 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 msgr2=0x7f64780b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6478089390 tx=0x7f646c016000 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 shutdown_connections 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6464066f30 0x7f64640693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f647815cc20 0x7f64781536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64780cd870 0x7f64780c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 --2- 172.21.15.16:0/2957909797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64780cf280 0x7f64780b4e90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 >> 172.21.15.16:0/2957909797 conn(0x7f6478099b40 msgr2=0x7f64780c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 shutdown_connections 2023-10-06T11:42:01.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.813+0000 7f647f0ed700 1 -- 172.21.15.16:0/2957909797 wait complete. 2023-10-06T11:42:01.828 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T11:42:01.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:01.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.945+0000 7f42d3a92700 1 Processor -- start 2023-10-06T11:42:01.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 -- start start 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f42cc15cc20 0x7f42cc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 0x7f42cc0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42cc0cdb10 con 0x7f42cc0cf280 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0b3110 con 0x7f42cc15cc20 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.946+0000 7f42d3a92700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0ccb20 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f42cc15cc20 0x7f42cc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d102d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 0x7f42cc0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f42cc15cc20 0x7f42cc0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:44984/0 (socket says 172.21.15.16:44984) 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d202f700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f42cc0cf280 0x7f42cc15d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48674/0 (socket says 172.21.15.16:48674) 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 -- 172.21.15.16:0/347638780 learned_addr learned my addr 172.21.15.16:0/347638780 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d102d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 0x7f42cc0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41492/0 (socket says 172.21.15.16:41492) 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 -- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 msgr2=0x7f42cc0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 --2- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 0x7f42cc0ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 -- 172.21.15.16:0/347638780 >> v1:172.21.15.177:6789/0 conn(0x7f42cc0cf280 legacy=0x7f42cc15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 -- 172.21.15.16:0/347638780 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0cbf10 con 0x7f42cc15cc20 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d082c700 1 -- 172.21.15.16:0/347638780 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929500074 0 0) 0x7f42cc0cdb10 con 0x7f42cc0cf280 2023-10-06T11:42:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.947+0000 7f42d182e700 1 --2- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc15cc20 0x7f42cc0cd160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0099b0 tx=0x7f42bc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f706862dbfd175 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d082c700 1 -- 172.21.15.16:0/347638780 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc009ea0 con 0x7f42cc15cc20 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d082c700 1 -- 172.21.15.16:0/347638780 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc00b860 con 0x7f42cc15cc20 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d082c700 1 -- 172.21.15.16:0/347638780 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00bc20 con 0x7f42cc15cc20 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 -- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc15cc20 msgr2=0x7f42cc0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 --2- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc15cc20 0x7f42cc0cd160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0099b0 tx=0x7f42bc009250 comp rx=0 tx=0).stop 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 -- 172.21.15.16:0/347638780 shutdown_connections 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 --2- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc15cc20 0x7f42cc0cd160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 --2- 172.21.15.16:0/347638780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc0cd6a0 0x7f42cc0ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 -- 172.21.15.16:0/347638780 >> 172.21.15.16:0/347638780 conn(0x7f42cc099b40 msgr2=0x7f42cc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 -- 172.21.15.16:0/347638780 shutdown_connections 2023-10-06T11:42:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.948+0000 7f42d3a92700 1 -- 172.21.15.16:0/347638780 wait complete. 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.949+0000 7f42d3a92700 1 Processor -- start 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.949+0000 7f42d3a92700 1 -- start start 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc0cf280 0x7f42cc0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 0x7f42cc0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0c8bb0 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0b6360 con 0x7f42cc0cf280 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d3a92700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc16d7f0 con 0x7f42cc15cc20 2023-10-06T11:42:01.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43016/0 (socket says 172.21.15.16:43016) 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 -- 172.21.15.16:0/3894107376 learned_addr learned my addr 172.21.15.16:0/3894107376 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d102d700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc0cf280 0x7f42cc0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d202f700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 0x7f42cc0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 -- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 msgr2=0x7f42cc0b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 0x7f42cc0b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 -- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc0cf280 msgr2=0x7f42cc0c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc0cf280 0x7f42cc0c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d182e700 1 -- 172.21.15.16:0/3894107376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0094f0 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.950+0000 7f42d202f700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 0x7f42cc0b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42d182e700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42bc006fd0 tx=0x7f42bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc01f070 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc000bc0 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc16dad0 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc00bc00 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.951+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc16e090 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.952+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc04c380 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.953+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f42bc023070 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.953+0000 7f42c9ffb700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 0x7f42b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.953+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f42bc086110 con 0x7f42cc0cd6a0 2023-10-06T11:42:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.954+0000 7f42d102d700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 0x7f42b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.954+0000 7f42d102d700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 0x7f42b80693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42cc16d370 tx=0x7f42c00091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:01.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:01.957+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42bc0532b0 con 0x7f42cc0cd6a0 2023-10-06T11:42:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.152+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42cc0b3500 con 0x7f42cc0cd6a0 2023-10-06T11:42:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.153+0000 7f42c9ffb700 1 -- 172.21.15.16:0/3894107376 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f42bc0568d0 con 0x7f42cc0cd6a0 2023-10-06T11:42:02.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:02.155 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:02.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 msgr2=0x7f42b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 0x7f42b80693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f42cc16d370 tx=0x7f42c00091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 msgr2=0x7f42cc0c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f42bc006fd0 tx=0x7f42bc017000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 shutdown_connections 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42cc0cd6a0 0x7f42cc0c7810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f42b8066f30 0x7f42b80693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f42cc15cc20 0x7f42cc0b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 --2- 172.21.15.16:0/3894107376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f42cc0cf280 0x7f42cc0c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 >> 172.21.15.16:0/3894107376 conn(0x7f42cc099b40 msgr2=0x7f42cc15fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 shutdown_connections 2023-10-06T11:42:02.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.155+0000 7f42d3a92700 1 -- 172.21.15.16:0/3894107376 wait complete. 2023-10-06T11:42:02.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T11:42:02.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T11:42:02.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T11:42:02.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T11:42:02.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T11:42:02.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T11:42:02.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T11:42:02.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T11:42:02.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.371+0000 7f4b5ee53700 1 Processor -- start 2023-10-06T11:42:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.371+0000 7f4b5ee53700 1 -- start start 2023-10-06T11:42:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.372+0000 7f4b5ee53700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4b581558f0 0x7f4b581568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.372+0000 7f4b5ee53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b58156e30 0x7f4b580c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.372+0000 7f4b5ee53700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5806af40 con 0x7f4b5806cf20 2023-10-06T11:42:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.372+0000 7f4b5ee53700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5806a610 con 0x7f4b58156e30 2023-10-06T11:42:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.372+0000 7f4b5ee53700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5806a8f0 con 0x7f4b581558f0 2023-10-06T11:42:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.373+0000 7f4b57fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b58156e30 0x7f4b580c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.373+0000 7f4b57fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4b58156e30 0x7f4b580c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43020/0 (socket says 172.21.15.16:43020) 2023-10-06T11:42:02.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.373+0000 7f4b57fff700 1 -- 172.21.15.16:0/402103498 learned_addr learned my addr 172.21.15.16:0/402103498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b57fff700 1 -- 172.21.15.16:0/402103498 >> v2:172.21.15.177:3300/0 conn(0x7f4b581558f0 msgr2=0x7f4b581568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b57fff700 1 --2- 172.21.15.16:0/402103498 >> v2:172.21.15.177:3300/0 conn(0x7f4b581558f0 0x7f4b581568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b57fff700 1 -- 172.21.15.16:0/402103498 >> v1:172.21.15.16:6789/0 conn(0x7f4b5806cf20 legacy=0x7f4b580c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b57fff700 1 -- 172.21.15.16:0/402103498 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5806abd0 con 0x7f4b58156e30 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b57fff700 1 --2- 172.21.15.16:0/402103498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b58156e30 0x7f4b580c8b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b5806dd30 tx=0x7f4b48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc94bdf3e06d181 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b577fe700 1 -- 172.21.15.16:0/402103498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800b820 con 0x7f4b58156e30 2023-10-06T11:42:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.374+0000 7f4b577fe700 1 -- 172.21.15.16:0/402103498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4800b980 con 0x7f4b58156e30 2023-10-06T11:42:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.375+0000 7f4b577fe700 1 -- 172.21.15.16:0/402103498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800bd20 con 0x7f4b58156e30 2023-10-06T11:42:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.375+0000 7f4b5ee53700 1 -- 172.21.15.16:0/402103498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b58156e30 msgr2=0x7f4b580c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.375+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/402103498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b58156e30 0x7f4b580c8b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b5806dd30 tx=0x7f4b48009250 comp rx=0 tx=0).stop 2023-10-06T11:42:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 -- 172.21.15.16:0/402103498 shutdown_connections 2023-10-06T11:42:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/402103498 >> v2:172.21.15.177:3300/0 conn(0x7f4b581558f0 0x7f4b581568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/402103498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b58156e30 0x7f4b580c8b70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 -- 172.21.15.16:0/402103498 >> 172.21.15.16:0/402103498 conn(0x7f4b5806c410 msgr2=0x7f4b5806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 -- 172.21.15.16:0/402103498 shutdown_connections 2023-10-06T11:42:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.376+0000 7f4b5ee53700 1 -- 172.21.15.16:0/402103498 wait complete. 2023-10-06T11:42:02.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.378+0000 7f4b5ee53700 1 Processor -- start 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.378+0000 7f4b5ee53700 1 -- start start 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b5806cf20 0x7f4b581560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b581558f0 0x7f4b581719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b58156e30 0x7f4b58171ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580cc8a0 con 0x7f4b581558f0 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580cc2e0 con 0x7f4b5806cf20 2023-10-06T11:42:02.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f4b5ee53700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580cc5c0 con 0x7f4b58156e30 2023-10-06T11:42:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f4b57fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b581558f0 0x7f4b581719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f4b57fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b581558f0 0x7f4b581719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43034/0 (socket says 172.21.15.16:43034) 2023-10-06T11:42:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f4b57fff700 1 -- 172.21.15.16:0/3448830142 learned_addr learned my addr 172.21.15.16:0/3448830142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f797413c700 1 Processor -- start 2023-10-06T11:42:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f1db5072700 1 Processor -- start 2023-10-06T11:42:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.379+0000 7f1db5072700 1 -- start start 2023-10-06T11:42:02.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f4b5cbef700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b5806cf20 0x7f4b581560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f4b5d3f0700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b58156e30 0x7f4b58171ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7fae12cb2700 1 Processor -- start 2023-10-06T11:42:02.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f3a0bd32700 1 Processor -- start 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.380+0000 7f797413c700 1 -- start start 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f797413c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f796c06b3d0 0x7f796c0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f797413c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c06af40 con 0x7f796c06cf20 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f797413c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c157000 con 0x7f796c1558f0 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f797413c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c06a610 con 0x7f796c06b3d0 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f79716d7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f796c06cf20 0x7f796c0c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56922/0 (socket says 172.21.15.16:56922) 2023-10-06T11:42:02.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f79716d7700 1 -- 172.21.15.16:0/2139848409 learned_addr learned my addr 172.21.15.16:0/2139848409 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 -- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b5806cf20 msgr2=0x7f4b581560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b5806cf20 0x7f4b581560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 -- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b581558f0 msgr2=0x7f4b581719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b581558f0 0x7f4b581719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b480094f0 con 0x7f4b58156e30 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f4b5d3f0700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b58156e30 0x7f4b58171ef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b4c0 tx=0x7f4b4c009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00f510 con 0x7f4b58156e30 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00f690 con 0x7f4b58156e30 2023-10-06T11:42:02.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00fa50 con 0x7f4b58156e30 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1db5072700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1db015e000 0x7f1db00c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1db5072700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 0x7f1db00c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1db5072700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00c6a00 con 0x7f1db015dc30 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1db5072700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00c9780 con 0x7f1db015e000 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1db5072700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00c6ca0 con 0x7f1db00c8dc0 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1daed9d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f1db015dc30 0x7f1db006ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48682/0 (socket says 172.21.15.16:48682) 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1daed9d700 1 -- 172.21.15.16:0/1397771132 learned_addr learned my addr 172.21.15.16:0/1397771132 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7f1daf59e700 1 --2- 172.21.15.16:0/1397771132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 0x7f1db00c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1dae59c700 1 --2- 172.21.15.16:0/1397771132 >> v2:172.21.15.16:3300/0 conn(0x7f1db015e000 0x7f1db00c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1daf59e700 1 -- 172.21.15.16:0/1397771132 >> v2:172.21.15.16:3300/0 conn(0x7f1db015e000 msgr2=0x7f1db00c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1daf59e700 1 --2- 172.21.15.16:0/1397771132 >> v2:172.21.15.16:3300/0 conn(0x7f1db015e000 0x7f1db00c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1daf59e700 1 -- 172.21.15.16:0/1397771132 >> v1:172.21.15.177:6789/0 conn(0x7f1db015dc30 legacy=0x7f1db006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1daf59e700 1 -- 172.21.15.16:0/1397771132 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00c6f80 con 0x7f1db00c8dc0 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1daf59e700 1 --2- 172.21.15.16:0/1397771132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 0x7f1db00c9240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e60 tx=0x7f1da8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cecd0110aed124e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f0caa607700 1 Processor -- start 2023-10-06T11:42:02.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.381+0000 7fae12cb2700 1 -- start start 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7fae12cb2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae0c06e440 0x7fae0c06e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7fae12cb2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c06a7d0 con 0x7fae0c06bab0 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7fae12cb2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c06bf30 con 0x7fae0c0ccce0 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7fae12cb2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c06b620 con 0x7fae0c06e440 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580c3610 con 0x7f4b58156e30 2023-10-06T11:42:02.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569382490 0 0) 0x7f796c06af40 con 0x7f796c06cf20 2023-10-06T11:42:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7958003570 con 0x7f796c06cf20 2023-10-06T11:42:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5816e380 con 0x7f4b58156e30 2023-10-06T11:42:02.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1dadd9b700 1 -- 172.21.15.16:0/1397771132 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800e070 con 0x7f1db00c8dc0 2023-10-06T11:42:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f1dadd9b700 1 -- 172.21.15.16:0/1397771132 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800bcc0 con 0x7f1db00c8dc0 2023-10-06T11:42:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f1dadd9b700 1 -- 172.21.15.16:0/1397771132 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8012680 con 0x7f1db00c8dc0 2023-10-06T11:42:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7fae10a4e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae0c06e440 0x7fae0c06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7fae10a4e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fae0c06e440 0x7fae0c06e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45012/0 (socket says 172.21.15.16:45012) 2023-10-06T11:42:02.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7fae10a4e700 1 -- 172.21.15.16:0/1741020465 learned_addr learned my addr 172.21.15.16:0/1741020465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677226377 0 0) 0x7f7958003570 con 0x7f796c06cf20 2023-10-06T11:42:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f796c06b3d0 msgr2=0x7f796c0c6a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f796c06b3d0 0x7f796c0c6a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 >> v1:172.21.15.177:6789/0 conn(0x7f796c1558f0 legacy=0x7f796c1568f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c06a8f0 con 0x7f796c06cf20 2023-10-06T11:42:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f7968001470 con 0x7f796c06cf20 2023-10-06T11:42:02.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f3a0bd32700 1 -- start start 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 >> v1:172.21.15.16:6789/0 conn(0x7f796c06cf20 legacy=0x7f796c0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958003b10 0x7f7958003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7958004810 0x7f7958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79580053c0 0x7f795800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c06a610 con 0x7f7958004810 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795800c230 con 0x7f79580053c0 2023-10-06T11:42:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795800c3f0 con 0x7f7958003b10 2023-10-06T11:42:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7fd5f1445700 1 Processor -- start 2023-10-06T11:42:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a0bd32700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3a0406e440 0x7f3a0406e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a0bd32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a040ccce0 0x7f3a0406bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a0bd32700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0406b620 con 0x7f3a0406a0c0 2023-10-06T11:42:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a0bd32700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0406aa10 con 0x7f3a040ccce0 2023-10-06T11:42:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a0bd32700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0406acf0 con 0x7f3a0406e440 2023-10-06T11:42:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f79716d7700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7958004810 0x7f7958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a092cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a040ccce0 0x7f3a0406bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a092cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a040ccce0 0x7f3a0406bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43068/0 (socket says 172.21.15.16:43068) 2023-10-06T11:42:02.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a092cd700 1 -- 172.21.15.16:0/3698362138 learned_addr learned my addr 172.21.15.16:0/3698362138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f79726d9700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79580053c0 0x7f795800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 -- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958003b10 msgr2=0x7f7958003f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958003b10 0x7f7958003f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 -- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79580053c0 msgr2=0x7f795800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79580053c0 0x7f795800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 -- 172.21.15.16:0/2139848409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958003570 con 0x7f7958004810 2023-10-06T11:42:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 -- 172.21.15.16:0/1397771132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 msgr2=0x7f1db00c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 --2- 172.21.15.16:0/1397771132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 0x7f1db00c9240 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009e60 tx=0x7f1da8009780 comp rx=0 tx=0).stop 2023-10-06T11:42:02.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f3a09ace700 1 --2- 172.21.15.16:0/3698362138 >> v2:172.21.15.177:3300/0 conn(0x7f3a0406e440 0x7f3a0406e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f79716d7700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7958004810 0x7f7958004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796800c2a0 tx=0x7f7968001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debe69fad3498886 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 -- 172.21.15.16:0/1397771132 shutdown_connections 2023-10-06T11:42:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 --2- 172.21.15.16:0/1397771132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db00c8dc0 0x7f1db00c9240 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 --2- 172.21.15.16:0/1397771132 >> v2:172.21.15.16:3300/0 conn(0x7f1db015e000 0x7f1db00c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 -- 172.21.15.16:0/1397771132 >> 172.21.15.16:0/1397771132 conn(0x7f1db006c410 msgr2=0x7f1db006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a092cd700 1 -- 172.21.15.16:0/3698362138 >> v2:172.21.15.177:3300/0 conn(0x7f3a0406e440 msgr2=0x7f3a0406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a092cd700 1 --2- 172.21.15.16:0/3698362138 >> v2:172.21.15.177:3300/0 conn(0x7f3a0406e440 0x7f3a0406e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a092cd700 1 -- 172.21.15.16:0/3698362138 >> v1:172.21.15.16:6789/0 conn(0x7f3a0406a0c0 legacy=0x7f3a0415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 -- 172.21.15.16:0/1397771132 shutdown_connections 2023-10-06T11:42:02.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f1db5072700 1 -- 172.21.15.16:0/1397771132 wait complete. 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.382+0000 7f0caa607700 1 -- start start 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f0caa607700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 0x7f0ca40cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f0caa607700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ca406cbf0 0x7f0ca406d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f0caa607700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca406d5a0 con 0x7f0ca40cd190 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f0caa607700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca4154580 con 0x7f0ca406cbf0 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.383+0000 7f0caa607700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40cf060 con 0x7f0ca415ab50 2023-10-06T11:42:02.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7fd5f1445700 1 -- start start 2023-10-06T11:42:02.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b58063e70 con 0x7f4b58156e30 2023-10-06T11:42:02.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f0ca8ba4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ca40cd190 0x7f0ca40cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56944/0 (socket says 172.21.15.16:56944) 2023-10-06T11:42:02.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f0ca8ba4700 1 -- 172.21.15.16:0/1566743927 learned_addr learned my addr 172.21.15.16:0/1566743927 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f0ca3fff700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 0x7f0ca40cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a092cd700 1 -- 172.21.15.16:0/3698362138 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0406afd0 con 0x7f3a040ccce0 2023-10-06T11:42:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a092cd700 1 --2- 172.21.15.16:0/3698362138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a040ccce0 0x7f3a0406bab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a04069960 tx=0x7f39fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a58658e9986477c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a08acc700 1 -- 172.21.15.16:0/3698362138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00e070 con 0x7f3a040ccce0 2023-10-06T11:42:02.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/1566743927 >> v2:172.21.15.177:3300/0 conn(0x7f0ca406cbf0 0x7f0ca406d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a08acc700 1 -- 172.21.15.16:0/3698362138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc00ba10 con 0x7f3a040ccce0 2023-10-06T11:42:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.384+0000 7f3a08acc700 1 -- 172.21.15.16:0/3698362138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0092e0 con 0x7f3a040ccce0 2023-10-06T11:42:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165549098 0 0) 0x7fadfc001110 con 0x7fae0c06bab0 2023-10-06T11:42:02.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf4003570 con 0x7fae0c06bab0 2023-10-06T11:42:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 -- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 msgr2=0x7f0ca40cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 0x7f0ca40cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 -- 172.21.15.16:0/1566743927 >> v1:172.21.15.16:6789/0 conn(0x7f0ca40cd190 legacy=0x7f0ca40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 -- 172.21.15.16:0/1566743927 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca40cf900 con 0x7f0ca406cbf0 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca3fff700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 0x7f0ca40cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca406d060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c980098e0 tx=0x7f0c980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23cd21525183d16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1950909403 0 0) 0x7fadf4003570 con 0x7fae0c06bab0 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 >> v2:172.21.15.177:3300/0 conn(0x7fae0c06e440 msgr2=0x7fae0c06e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 --2- 172.21.15.16:0/1741020465 >> v2:172.21.15.177:3300/0 conn(0x7fae0c06e440 0x7fae0c06e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 >> v1:172.21.15.177:6789/0 conn(0x7fae0c0ccce0 legacy=0x7fae0c06a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c06aa10 con 0x7fae0c06bab0 2023-10-06T11:42:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fadfc001600 con 0x7fae0c06bab0 2023-10-06T11:42:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3698362138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a040ccce0 msgr2=0x7f3a0406bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3698362138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a040ccce0 0x7f3a0406bab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a04069960 tx=0x7f39fc009840 comp rx=0 tx=0).stop 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 >> v1:172.21.15.16:6789/0 conn(0x7fae0c06bab0 legacy=0x7fae0c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 0x7fadf4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadf4004810 0x7fadf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4005210 0x7fadf400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06b620 con 0x7fadf4003ab0 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf400c1b0 con 0x7fadf4004810 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf400c370 con 0x7fadf4005210 2023-10-06T11:42:02.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3698362138 shutdown_connections 2023-10-06T11:42:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3698362138 >> v2:172.21.15.177:3300/0 conn(0x7f3a0406e440 0x7f3a0406e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3698362138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a040ccce0 0x7f3a0406bab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3698362138 >> 172.21.15.16:0/3698362138 conn(0x7f3a0406cee0 msgr2=0x7f3a040cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3698362138 shutdown_connections 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800cbd0 con 0x7f7958004810 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800cd30 con 0x7f7958004810 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f7970ed6700 1 -- 172.21.15.16:0/2139848409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800cea0 con 0x7f7958004810 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4b4c01a400 con 0x7f4b58156e30 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7f4b54ff9700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b44067080 0x7f4b44069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4b4c084070 con 0x7f4b58156e30 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b54ff9700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f4b4406f8b0 0x7f4b44071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b44072370 con 0x7f4b4406f8b0 2023-10-06T11:42:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b4c0511e0 con 0x7f4b58156e30 2023-10-06T11:42:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b5cbef700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b44067080 0x7f4b44069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b57fff700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f4b4406f8b0 0x7f4b44071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f1db5072700 1 Processor -- start 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.385+0000 7fd5f1445700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5f1445700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec06ac40 con 0x7fd5ec06a7d0 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5f1445700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec06af20 con 0x7fd5ec06bce0 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5f1445700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec06b200 con 0x7fd5ec06e440 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41568/0 (socket says 172.21.15.16:41568) 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 -- 172.21.15.16:0/2613018880 learned_addr learned my addr 172.21.15.16:0/2613018880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 -- 172.21.15.16:0/2613018880 >> v1:172.21.15.177:6789/0 conn(0x7fd5ec06bce0 legacy=0x7fd5ec06a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 -- 172.21.15.16:0/2613018880 >> v1:172.21.15.16:6789/0 conn(0x7fd5ec06a7d0 legacy=0x7fd5ec15ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7fd5eaffd700 1 -- 172.21.15.16:0/2613018880 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec06b4e0 con 0x7fd5ec06e440 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f0ca2ffd700 1 -- 172.21.15.16:0/1566743927 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b640 con 0x7f0ca406cbf0 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f0ca2ffd700 1 -- 172.21.15.16:0/1566743927 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800b7a0 con 0x7f0ca406cbf0 2023-10-06T11:42:02.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f0ca2ffd700 1 -- 172.21.15.16:0/1566743927 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800bb40 con 0x7f0ca406cbf0 2023-10-06T11:42:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f0caa607700 1 -- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 msgr2=0x7f0ca406d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f0caa607700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca406d060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0c980098e0 tx=0x7f0c980091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 0x7fadf4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae1124f700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4005210 0x7fadf400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae0bfff700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadf4004810 0x7fadf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f1db5072700 1 -- start start 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1db00c8dc0 0x7f1db00cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 0x7f1db00ab590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015e000 0x7f1db00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00abc70 con 0x7f1db015e000 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00abf50 con 0x7f1db00c8dc0 2023-10-06T11:42:02.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1db5072700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00ac230 con 0x7f1db015dc30 2023-10-06T11:42:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 -- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4005210 msgr2=0x7fadf400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4005210 0x7fadf400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 -- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadf4004810 msgr2=0x7fadf4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fadf4004810 0x7fadf4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 -- 172.21.15.16:0/1741020465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4003570 con 0x7fadf4003ab0 2023-10-06T11:42:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae1124f700 1 -- 172.21.15.16:0/1741020465 reap_dead start 2023-10-06T11:42:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/2613018880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00099e0 tx=0x7fd5e0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=878713234bd8c7ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3698362138 wait complete. 2023-10-06T11:42:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1daf59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015e000 0x7f1db00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 0x7f1db00ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1dae59c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 0x7f1db00ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41572/0 (socket says 172.21.15.16:41572) 2023-10-06T11:42:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1dae59c700 1 -- 172.21.15.16:0/2863706176 learned_addr learned my addr 172.21.15.16:0/2863706176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f1daed9d700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1db00c8dc0 0x7f1db00cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7fae10a4e700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 0x7fadf4003ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae0400e2d0 tx=0x7fae0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38bfa48678cec5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.386+0000 7f4b5cbef700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b44067080 0x7f4b44069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001250 tx=0x7f4b50009620 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f4b57fff700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f4b4406f8b0 0x7f4b44071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f3a0bd32700 1 Processor -- start 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f797413c700 1 -- 172.21.15.16:0/2139848409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7958004810 msgr2=0x7f7958004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f797413c700 1 --2- 172.21.15.16:0/2139848409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7958004810 0x7f7958004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f796800c2a0 tx=0x7f7968001e00 comp rx=0 tx=0).stop 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.387+0000 7f79726d9700 1 -- 172.21.15.16:0/2139848409 reap_dead start 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/2613018880 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000b820 con 0x7fd5ec06e440 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/2613018880 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e000b980 con 0x7fd5ec06e440 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/2613018880 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000bd20 con 0x7fd5ec06e440 2023-10-06T11:42:02.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== osd.2 v2:172.21.15.16:6819/90626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4b44072370 con 0x7f4b4406f8b0 2023-10-06T11:42:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 -- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1db00c8dc0 msgr2=0x7f1db00cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1db00c8dc0 0x7f1db00cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 -- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015e000 msgr2=0x7f1db00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015e000 0x7f1db00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 -- 172.21.15.16:0/2863706176 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8009a20 con 0x7f1db015dc30 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f0caa607700 1 -- 172.21.15.16:0/1566743927 shutdown_connections 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f0caa607700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca415ab50 0x7f0ca40cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f0caa607700 1 --2- 172.21.15.16:0/1566743927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca406d060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f0caa607700 1 -- 172.21.15.16:0/1566743927 >> 172.21.15.16:0/1566743927 conn(0x7f0ca408a270 msgr2=0x7f0ca4099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f3a0bd32700 1 -- start start 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a0406a0c0 0x7f3a0415bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0406e440 0x7f3a040ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a040ccce0 0x7f3a040c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040ad4b0 con 0x7f3a0406a0c0 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040ac8a0 con 0x7f3a0406e440 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0bd32700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a040acb80 con 0x7f3a040ccce0 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f797413c700 1 -- 172.21.15.16:0/2139848409 shutdown_connections 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f797413c700 1 -- 172.21.15.16:0/2139848409 >> 172.21.15.16:0/2139848409 conn(0x7f796c06c410 msgr2=0x7f796c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04018070 con 0x7fadf4003ab0 2023-10-06T11:42:02.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400f040 con 0x7fadf4003ab0 2023-10-06T11:42:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7fae0b7fe700 1 -- 172.21.15.16:0/1741020465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04009320 con 0x7fadf4003ab0 2023-10-06T11:42:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 -- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 msgr2=0x7fadf4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 0x7fadf4003ec0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae0400e2d0 tx=0x7fae0400b040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fd5f1445700 1 -- 172.21.15.16:0/2613018880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 msgr2=0x7fd5ec06e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fd5f1445700 1 --2- 172.21.15.16:0/2613018880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00099e0 tx=0x7fd5e0009250 comp rx=0 tx=0).stop 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 -- 172.21.15.16:0/1741020465 shutdown_connections 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 --2- 172.21.15.16:0/1741020465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadf4003ab0 0x7fadf4003ec0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 -- 172.21.15.16:0/1741020465 >> 172.21.15.16:0/1741020465 conn(0x7fae0c06cee0 msgr2=0x7fae0c0cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a092cd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0406e440 0x7f3a040ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a092cd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0406e440 0x7f3a040ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45060/0 (socket says 172.21.15.16:45060) 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a092cd700 1 -- 172.21.15.16:0/3314258996 learned_addr learned my addr 172.21.15.16:0/3314258996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fd5f1445700 1 -- 172.21.15.16:0/2613018880 shutdown_connections 2023-10-06T11:42:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fd5f1445700 1 --2- 172.21.15.16:0/2613018880 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06e440 0x7fd5ec06e810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fd5f1445700 1 -- 172.21.15.16:0/2613018880 >> 172.21.15.16:0/2613018880 conn(0x7fd5ec06cee0 msgr2=0x7fd5ec0cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a0a2cf700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a040ccce0 0x7f3a040c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f3a09ace700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a0406a0c0 0x7f3a0415bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f797413c700 1 -- 172.21.15.16:0/2139848409 shutdown_connections 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1daf59e700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1db015e000 0x7f1db00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1dae59c700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 0x7f1db00ab590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d890 tx=0x7f1da4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da40125f0 con 0x7f1db015dc30 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da4012750 con 0x7f1db015dc30 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db00ac510 con 0x7f1db015dc30 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da4012ad0 con 0x7f1db015dc30 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.388+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db00cc970 con 0x7f1db015dc30 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f0caa607700 1 -- 172.21.15.16:0/1566743927 shutdown_connections 2023-10-06T11:42:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7f0caa607700 1 -- 172.21.15.16:0/1566743927 wait complete. 2023-10-06T11:42:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 -- 172.21.15.16:0/1741020465 shutdown_connections 2023-10-06T11:42:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.389+0000 7fae12cb2700 1 -- 172.21.15.16:0/1741020465 wait complete. 2023-10-06T11:42:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7fd5f1445700 1 -- 172.21.15.16:0/2613018880 shutdown_connections 2023-10-06T11:42:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 -- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a040ccce0 msgr2=0x7f3a040c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a040ccce0 0x7f3a040c3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 -- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0406e440 msgr2=0x7f3a040ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a0406e440 0x7f3a040ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 -- 172.21.15.16:0/3314258996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc009ae0 con 0x7f3a0406a0c0 2023-10-06T11:42:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7fd5f1445700 1 -- 172.21.15.16:0/2613018880 wait complete. 2023-10-06T11:42:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f3a09ace700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a0406a0c0 0x7f3a0415bd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39f400d890 tx=0x7f39f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f797413c700 1 -- 172.21.15.16:0/2139848409 wait complete. 2023-10-06T11:42:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f0caa607700 1 Processor -- start 2023-10-06T11:42:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.390+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1da401e460 con 0x7f1db015dc30 2023-10-06T11:42:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f1d96ffd700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d9c066fb0 0x7f1d9c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1da4084d40 con 0x7f1db015dc30 2023-10-06T11:42:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7fae12cb2700 1 Processor -- start 2023-10-06T11:42:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40125f0 con 0x7f3a0406a0c0 2023-10-06T11:42:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fd5f1445700 1 Processor -- start 2023-10-06T11:42:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f4012770 con 0x7f3a0406a0c0 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f4012b30 con 0x7f3a0406a0c0 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a040ace60 con 0x7f3a0406a0c0 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f1daed9d700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d9c066fb0 0x7f1d9c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7fae12cb2700 1 -- start start 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fae12cb2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4003ab0 0x7fae0c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fae12cb2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0c160ff0 0x7fae0c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fae12cb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0c2220 0x7fae0c0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fae12cb2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0c2be0 con 0x7fae0c0c2220 2023-10-06T11:42:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae12cb2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0abd70 con 0x7fae0c160ff0 2023-10-06T11:42:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae12cb2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c162390 con 0x7fadf4003ab0 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.391+0000 7f0caa607700 1 -- start start 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca40cd190 0x7f0ca40c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca415ab50 0x7f0ca4152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b4af0 con 0x7f0ca415ab50 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b6660 con 0x7f0ca406cbf0 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f0caa607700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40c75c0 con 0x7f0ca40cd190 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca40cd190 0x7f0ca40c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca8ba4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca415ab50 0x7f0ca4152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca8ba4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca415ab50 0x7f0ca4152ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43106/0 (socket says 172.21.15.16:43106) 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca8ba4700 1 -- 172.21.15.16:0/3852820252 learned_addr learned my addr 172.21.15.16:0/3852820252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 -- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 msgr2=0x7f0ca40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ca406cbf0 0x7f0ca40c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 -- 172.21.15.16:0/3852820252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca415ab50 msgr2=0x7f0ca4152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ca415ab50 0x7f0ca4152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f0ca37fe700 1 -- 172.21.15.16:0/3852820252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c940093c0 con 0x7f0ca40cd190 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f797413c700 1 Processor -- start 2023-10-06T11:42:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f797413c700 1 -- start start 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958004810 0x7f796c0cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f796c1580c0 0x7f796c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 0x7f796c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1591b0 con 0x7f796c154cb0 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c1585a0 con 0x7f796c1580c0 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7f797413c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c158880 con 0x7f7958004810 2023-10-06T11:42:02.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 0x7f796c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae0bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0c160ff0 0x7fae0c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae0bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0c160ff0 0x7fae0c161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45066/0 (socket says 172.21.15.16:45066) 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae0bfff700 1 -- 172.21.15.16:0/3444614613 learned_addr learned my addr 172.21.15.16:0/3444614613 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0c2220 0x7fae0c0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae10a4e700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4003ab0 0x7fae0c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 -- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4003ab0 msgr2=0x7fae0c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fadf4003ab0 0x7fae0c15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 -- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0c160ff0 msgr2=0x7fae0c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae0c160ff0 0x7fae0c161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fae1124f700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c162670 con 0x7fae0c0c2220 2023-10-06T11:42:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 0x7f796c0cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43122/0 (socket says 172.21.15.16:43122) 2023-10-06T11:42:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 -- 172.21.15.16:0/422827519 learned_addr learned my addr 172.21.15.16:0/422827519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0ca37fe700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca40cd190 0x7f0ca40c4860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c98016000 tx=0x7f0c98017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fae1124f700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0c2220 0x7fae0c0c26a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0715f0 tx=0x7fadfc0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f7971ed8700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958004810 0x7f796c0cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f1daed9d700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d9c066fb0 0x7f1d9c069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f1db5072700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f1db00acd10 0x7f1db00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 --> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1db015d3f0 con 0x7f1db00acd10 2023-10-06T11:42:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7fd5f1445700 1 -- start start 2023-10-06T11:42:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ec06a7d0 0x7fd5ec16ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 0x7fd5ec159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5ec06e440 0x7fd5ec171690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec152df0 con 0x7fd5ec06a7d0 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec159a40 con 0x7fd5ec06e440 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.393+0000 7fd5f1445700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec172200 con 0x7fd5ec06bce0 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fd5ea7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 0x7fd5ec159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fd5ea7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 0x7fd5ec159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41604/0 (socket says 172.21.15.16:41604) 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fd5ea7fc700 1 -- 172.21.15.16:0/837572392 learned_addr learned my addr 172.21.15.16:0/837572392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00d820 con 0x7fae0c0c2220 2023-10-06T11:42:02.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc00d9a0 con 0x7fae0c0c2220 2023-10-06T11:42:02.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9801bd70 con 0x7f0ca40cd190 2023-10-06T11:42:02.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00dd60 con 0x7fae0c0c2220 2023-10-06T11:42:02.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c98011420 con 0x7f0ca40cd190 2023-10-06T11:42:02.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c980117e0 con 0x7f0ca40cd190 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fd5eb7fe700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5ec06e440 0x7fd5ec171690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.392+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0415c250 con 0x7f3a0406a0c0 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 -- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958004810 msgr2=0x7f796c0cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7958004810 0x7f796c0cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 -- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f796c1580c0 msgr2=0x7f796c154770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f796c1580c0 0x7f796c154770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f79726d9700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c158b60 con 0x7f796c154cb0 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f1daf59e700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f1db00acd10 0x7f1db00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ec06a7d0 0x7fd5ec16ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 -- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5ec06e440 msgr2=0x7fd5ec171690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5ec06e440 0x7fd5ec171690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 -- 172.21.15.16:0/837572392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ec06a7d0 msgr2=0x7fd5ec16ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5ec06a7d0 0x7fd5ec16ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00094f0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98009420 con 0x7f0ca40cd190 2023-10-06T11:42:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.394+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca40c65a0 con 0x7f0ca40cd190 2023-10-06T11:42:02.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c162950 con 0x7fae0c0c2220 2023-10-06T11:42:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c161f60 con 0x7fae0c0c2220 2023-10-06T11:42:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7fd5ea7fc700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 0x7fd5ec159f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c7c0 tx=0x7fd5dc007f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f39f4013480 con 0x7f3a0406a0c0 2023-10-06T11:42:02.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7f39f9ffb700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f39e8067000 0x7f39e80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7f79726d9700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 0x7f796c0cb290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f796c06b830 tx=0x7f79640097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.395+0000 7f3a092cd700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f39e8067000 0x7f39e80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f1daf59e700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f1db00acd10 0x7f1db00ad0e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae0c063e70 con 0x7fae0c0c2220 2023-10-06T11:42:02.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796400be90 con 0x7f796c154cb0 2023-10-06T11:42:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7964015410 con 0x7f796c154cb0 2023-10-06T11:42:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79640157d0 con 0x7f796c154cb0 2023-10-06T11:42:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c158e40 con 0x7f796c154cb0 2023-10-06T11:42:02.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c98011bd0 con 0x7f0ca40cd190 2023-10-06T11:42:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c0cb7d0 con 0x7f796c154cb0 2023-10-06T11:42:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.396+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f39f4012ea0 con 0x7f3a0406a0c0 2023-10-06T11:42:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f0c8bfff700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c90067080 0x7f0c90069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== osd.3 v2:172.21.15.16:6816/90633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1db015d3f0 con 0x7f1db00acd10 2023-10-06T11:42:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c980a7910 con 0x7f0ca40cd190 2023-10-06T11:42:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f7f8f586700 1 Processor -- start 2023-10-06T11:42:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc011590 con 0x7fd5ec06bce0 2023-10-06T11:42:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc0116f0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc011ab0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec1724e0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec16e6c0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f796c063e70 con 0x7f796c154cb0 2023-10-06T11:42:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7f7f8f586700 1 -- start start 2023-10-06T11:42:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.397+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadfc014680 con 0x7fae0c0c2220 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae08ff9700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fadf4068cf0 0x7fadf406b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fadfc084cd0 con 0x7fae0c0c2220 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae10a4e700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fadf4068cf0 0x7fadf406b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae08ff9700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7fadf40715a0 0x7fadf40739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadf4074060 con 0x7fadf40715a0 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadfc085000 con 0x7fae0c0c2220 2023-10-06T11:42:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.398+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd5cc000fd0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7964015c00 con 0x7f796c154cb0 2023-10-06T11:42:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f0ca3fff700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c90067080 0x7f0c90069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8f586700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 0x7f7f880b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8f586700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880b0d50 con 0x7f7f880b40b0 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8f586700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880b0140 con 0x7f7f880b6360 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8f586700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880b0420 con 0x7f7f88159800 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8db23700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f880b40b0 0x7f7f880cf830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56974/0 (socket says 172.21.15.16:56974) 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8db23700 1 -- 172.21.15.16:0/1945777993 learned_addr learned my addr 172.21.15.16:0/1945777993 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7fae0bfff700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7fadf40715a0 0x7fadf40739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7fae10a4e700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fadf4068cf0 0x7fadf406b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae04009560 tx=0x7fae04018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f8d322700 1 --2- 172.21.15.16:0/1945777993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 0x7f7f880b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7961ffb700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7950067080 0x7f7950069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5dc01b3b0 con 0x7fd5ec06bce0 2023-10-06T11:42:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f0caa607700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f0ca40b4350 0x7f0ca4157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 --> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ca40c6cd0 con 0x7f0ca40b4350 2023-10-06T11:42:02.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7964084fb0 con 0x7f796c154cb0 2023-10-06T11:42:02.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952105444 0 0) 0x7f7f880b0d50 con 0x7f7f880b40b0 2023-10-06T11:42:02.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f74003570 con 0x7f7f880b40b0 2023-10-06T11:42:02.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7fae0bfff700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7fadf40715a0 0x7fadf40739b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.399+0000 7f7961ffb700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f795006f930 0x7f7950071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79500723f0 con 0x7f795006f930 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79640852e0 con 0x7f796c154cb0 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f8d322700 1 -- 172.21.15.16:0/1945777993 >> v1:172.21.15.177:6789/0 conn(0x7f7f880b6360 legacy=0x7f7f880b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802764022 0 0) 0x7f7f880b0140 con 0x7f7f880b6360 2023-10-06T11:42:02.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f0ca3fff700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c90067080 0x7f0c90069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006ea0 tx=0x7f0c94008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f8d322700 1 -- 172.21.15.16:0/1945777993 >> v1:172.21.15.16:6789/0 conn(0x7f7f880b40b0 legacy=0x7f7f880cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f8d322700 1 -- 172.21.15.16:0/1945777993 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880b0700 con 0x7f7f88159800 2023-10-06T11:42:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f0ca8ba4700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f0ca40b4350 0x7f0ca4157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f3a092cd700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f39e8067000 0x7f39e80694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a041615e0 tx=0x7f39fc017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f79716d7700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f795006f930 0x7f7950071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7971ed8700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7950067080 0x7f7950069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5d2ffd700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5d8067080 0x7fd5d8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7f39e40015f0 0x7f39e4003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 --> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39e4006b90 con 0x7f39e40015f0 2023-10-06T11:42:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f8d322700 1 --2- 172.21.15.16:0/1945777993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 0x7f7f880b2a70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009e90 tx=0x7f7f80009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0c90f09fce4a99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5dc083c30 con 0x7fd5ec06bce0 2023-10-06T11:42:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5d8067080 0x7fd5d8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000e070 con 0x7f7f88159800 2023-10-06T11:42:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8000bc30 con 0x7f7f88159800 2023-10-06T11:42:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f7f7ffff700 1 -- 172.21.15.16:0/1945777993 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80012430 con 0x7f7f88159800 2023-10-06T11:42:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5d2ffd700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd5d806f930 0x7fd5d8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f0ca8ba4700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f0ca40b4350 0x7f0ca4157e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd5d80723f0 con 0x7fd5d806f930 2023-10-06T11:42:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc088220 con 0x7fd5ec06bce0 2023-10-06T11:42:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f79716d7700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f795006f930 0x7f7950071d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.400+0000 7f3a0a2cf700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7f39e40015f0 0x7f39e4003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7971ed8700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7950067080 0x7f7950069530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f795c006fd0 tx=0x7f795c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7fd5eb7fe700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd5d806f930 0x7fd5d8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7fd5eaffd700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5d8067080 0x7fd5d8069530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00099e0 tx=0x7fd5e0016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f3a0a2cf700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7f39e40015f0 0x7f39e4003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7fd5eb7fe700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd5d806f930 0x7fd5d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== osd.7 v2:172.21.15.177:6816/90844 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fadf4074060 con 0x7fadf40715a0 2023-10-06T11:42:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== osd.6 v2:172.21.15.177:6824/90887 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0ca40c6cd0 con 0x7f0ca40b4350 2023-10-06T11:42:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7f8f586700 1 -- 172.21.15.16:0/1945777993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 msgr2=0x7f7f880b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7f8f586700 1 --2- 172.21.15.16:0/1945777993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 0x7f7f880b2a70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009e90 tx=0x7f7f80009780 comp rx=0 tx=0).stop 2023-10-06T11:42:02.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7f8f586700 1 -- 172.21.15.16:0/1945777993 shutdown_connections 2023-10-06T11:42:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7f8f586700 1 --2- 172.21.15.16:0/1945777993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f88159800 0x7f7f880b2a70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7f8f586700 1 -- 172.21.15.16:0/1945777993 >> 172.21.15.16:0/1945777993 conn(0x7f7f8808a230 msgr2=0x7f7f88099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.401+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== osd.1 v2:172.21.15.16:6800/90616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f79500723f0 con 0x7f795006f930 2023-10-06T11:42:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.402+0000 7f7f8f586700 1 -- 172.21.15.16:0/1945777993 shutdown_connections 2023-10-06T11:42:02.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.402+0000 7f7f8f586700 1 -- 172.21.15.16:0/1945777993 wait complete. 2023-10-06T11:42:02.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.402+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== osd.0 v2:172.21.15.16:6808/90625 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f39e4006b90 con 0x7f39e40015f0 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.403+0000 7f7f8f586700 1 Processor -- start 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.404+0000 7f7f8f586700 1 -- start start 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f880b40b0 0x7f7f881622a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f880b6360 0x7f7f88164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 0x7f7f88158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8816e070 con 0x7f7f880b6360 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f881649c0 con 0x7f7f88159800 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8f586700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f881628e0 con 0x7f7f880b40b0 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8db23700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 0x7f7f88158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8db23700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 0x7f7f88158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45090/0 (socket says 172.21.15.16:45090) 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.405+0000 7f7f8db23700 1 -- 172.21.15.16:0/19566943 learned_addr learned my addr 172.21.15.16:0/19566943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 -- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f880b40b0 msgr2=0x7f7f881622a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f880b40b0 0x7f7f881622a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 -- 172.21.15.16:0/19566943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f880b6360 msgr2=0x7f7f88164480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f880b6360 0x7f7f88164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 -- 172.21.15.16:0/19566943 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f80009a20 con 0x7f7f88159800 2023-10-06T11:42:02.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f8db23700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 0x7f7f88158e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f780010e0 tx=0x7f7f78007dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.407+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78005c00 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.408+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78005d60 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.408+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f88162bc0 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.408+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f88163180 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.408+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780072e0 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.410+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f78012400 con 0x7f7f88159800 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.410+0000 7f7f7d7fa700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.410+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f78083b60 con 0x7f7f88159800 2023-10-06T11:42:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.402+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== osd.4 v2:172.21.15.177:6808/90842 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd5d80723f0 con 0x7fd5d806f930 2023-10-06T11:42:02.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.412+0000 7f7f8f586700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f7f680015f0 0x7f7f68003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.412+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 --> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f68006b90 con 0x7f7f680015f0 2023-10-06T11:42:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.413+0000 7f7f8cb21700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f7f680015f0 0x7f7f68003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.413+0000 7f7f8cb21700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f7f680015f0 0x7f7f68003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.414+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 --> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b5806e060 con 0x7f4b4406f8b0 2023-10-06T11:42:02.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.414+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== osd.5 v2:172.21.15.177:6800/90843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7f68006b90 con 0x7f7f680015f0 2023-10-06T11:42:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.422+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 --> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae0c06a7d0 con 0x7fadf40715a0 2023-10-06T11:42:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.424+0000 7f7f8d322700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f6c066f30 0x7f7f6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.428+0000 7f7f8d322700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009de0 tx=0x7f7f80019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.430+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 --> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1db006e060 con 0x7f1db00acd10 2023-10-06T11:42:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.430+0000 7f1d96ffd700 1 -- 172.21.15.16:0/2863706176 <== osd.3 v2:172.21.15.16:6816/90633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1db006e060 con 0x7f1db00acd10 2023-10-06T11:42:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.431+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f1db00acd10 msgr2=0x7f1db00ad0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.431+0000 7f1db5072700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f1db00acd10 0x7f1db00ad0e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b54ff9700 1 -- 172.21.15.16:0/3448830142 <== osd.2 v2:172.21.15.16:6819/90626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b5806e060 con 0x7f4b4406f8b0 2023-10-06T11:42:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d9c066fb0 msgr2=0x7f1d9c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d9c066fb0 0x7f1d9c069460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1da0000c00 tx=0x7f1da0009040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 msgr2=0x7f1db00ab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 --2- 172.21.15.16:0/2863706176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1db015dc30 0x7f1db00ab590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1da400d890 tx=0x7f1da4008040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f4b4406f8b0 msgr2=0x7f4b44071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f4b4406f8b0 0x7f4b44071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1daf59e700 1 -- 172.21.15.16:0/2863706176 reap_dead start 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 shutdown_connections 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 >> 172.21.15.16:0/2863706176 conn(0x7f1db006c410 msgr2=0x7f1db00c2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b44067080 msgr2=0x7f4b44069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b44067080 0x7f4b44069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b50001250 tx=0x7f4b50009620 comp rx=0 tx=0).stop 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b58156e30 msgr2=0x7f4b58171ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 --2- 172.21.15.16:0/3448830142 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4b58156e30 0x7f4b58171ef0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b4c0 tx=0x7f4b4c009060 comp rx=0 tx=0).stop 2023-10-06T11:42:02.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5d3f0700 1 -- 172.21.15.16:0/3448830142 reap_dead start 2023-10-06T11:42:02.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 shutdown_connections 2023-10-06T11:42:02.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 shutdown_connections 2023-10-06T11:42:02.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 >> 172.21.15.16:0/3448830142 conn(0x7f4b5806c410 msgr2=0x7f4b580c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.432+0000 7f1db5072700 1 -- 172.21.15.16:0/2863706176 wait complete. 2023-10-06T11:42:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.433+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 --> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39e4005c80 con 0x7f39e40015f0 2023-10-06T11:42:02.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.433+0000 7f39f9ffb700 1 -- 172.21.15.16:0/3314258996 <== osd.0 v2:172.21.15.16:6808/90625 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39e4005c80 con 0x7f39e40015f0 2023-10-06T11:42:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.433+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 --> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f796c0c1660 con 0x7f795006f930 2023-10-06T11:42:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.433+0000 7f7961ffb700 1 -- 172.21.15.16:0/422827519 <== osd.1 v2:172.21.15.16:6800/90616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f796c0c1660 con 0x7f795006f930 2023-10-06T11:42:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7f39e40015f0 msgr2=0x7f39e4003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7f39e40015f0 0x7f39e4003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f39e8067000 msgr2=0x7f39e80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f39e8067000 0x7f39e80694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a041615e0 tx=0x7f39fc017000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a0406a0c0 msgr2=0x7f3a0415bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 --2- 172.21.15.16:0/3314258996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a0406a0c0 0x7f3a0415bd10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f39f400d890 tx=0x7f39f4008040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0a2cf700 1 -- 172.21.15.16:0/3314258996 reap_dead start 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 shutdown_connections 2023-10-06T11:42:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 >> 172.21.15.16:0/3314258996 conn(0x7f3a0406cee0 msgr2=0x7f3a040cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f795006f930 msgr2=0x7f7950071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.434+0000 7f797413c700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f795006f930 0x7f7950071d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.435+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 shutdown_connections 2023-10-06T11:42:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.436+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 --> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ca40c6300 con 0x7f0ca40b4350 2023-10-06T11:42:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 shutdown_connections 2023-10-06T11:42:02.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 --> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd5cc002d30 con 0x7fd5d806f930 2023-10-06T11:42:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae08ff9700 1 -- 172.21.15.16:0/3444614613 <== osd.7 v2:172.21.15.177:6816/90844 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fae0c06a7d0 con 0x7fadf40715a0 2023-10-06T11:42:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f3a0bd32700 1 -- 172.21.15.16:0/3314258996 wait complete. 2023-10-06T11:42:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7950067080 msgr2=0x7f7950069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7950067080 0x7f7950069530 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f795c006fd0 tx=0x7f795c008040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 msgr2=0x7f796c0cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 --2- 172.21.15.16:0/422827519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f796c154cb0 0x7f796c0cb290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f796c06b830 tx=0x7f79640097a0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f79726d9700 1 -- 172.21.15.16:0/422827519 reap_dead start 2023-10-06T11:42:02.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f0c8bfff700 1 -- 172.21.15.16:0/3852820252 <== osd.6 v2:172.21.15.177:6824/90887 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0c9c0020e0 con 0x7f0ca40b4350 2023-10-06T11:42:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 shutdown_connections 2023-10-06T11:42:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 >> 172.21.15.16:0/422827519 conn(0x7f796c06c410 msgr2=0x7f796c0c1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 shutdown_connections 2023-10-06T11:42:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fd5d2ffd700 1 -- 172.21.15.16:0/837572392 <== osd.4 v2:172.21.15.177:6808/90842 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd5cc002d30 con 0x7fd5d806f930 2023-10-06T11:42:02.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7f797413c700 1 -- 172.21.15.16:0/422827519 wait complete. 2023-10-06T11:42:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7fadf40715a0 msgr2=0x7fadf40739b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7fadf40715a0 0x7fadf40739b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fadf4068cf0 msgr2=0x7fadf406b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fadf4068cf0 0x7fadf406b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae04009560 tx=0x7fae04018000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0c2220 msgr2=0x7fae0c0c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.437+0000 7fae12cb2700 1 --2- 172.21.15.16:0/3444614613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae0c0c2220 0x7fae0c0c26a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0715f0 tx=0x7fadfc0078c0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd5d806f930 msgr2=0x7fd5d8071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd5d806f930 0x7fd5d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5d8067080 msgr2=0x7fd5d8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5d8067080 0x7fd5d8069530 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00099e0 tx=0x7fd5e0016000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 msgr2=0x7fd5ec159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5f1445700 1 --2- 172.21.15.16:0/837572392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5ec06bce0 0x7fd5ec159f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00c7c0 tx=0x7fd5dc007f50 comp rx=0 tx=0).stop 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fd5eb7fe700 1 -- 172.21.15.16:0/837572392 reap_dead start 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fae1124f700 1 -- 172.21.15.16:0/3444614613 reap_dead start 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 shutdown_connections 2023-10-06T11:42:02.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.438+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 >> 172.21.15.16:0/3444614613 conn(0x7fae0c06cee0 msgr2=0x7fae0c164cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f4b5ee53700 1 -- 172.21.15.16:0/3448830142 wait complete. 2023-10-06T11:42:02.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f0ca40b4350 msgr2=0x7f0ca4157e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f0ca40b4350 0x7f0ca4157e80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c90067080 msgr2=0x7f0c90069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c90067080 0x7f0c90069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c94006ea0 tx=0x7f0c94008040 comp rx=0 tx=0).stop 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca40cd190 msgr2=0x7f0ca40c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.439+0000 7f0caa607700 1 --2- 172.21.15.16:0/3852820252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ca40cd190 0x7f0ca40c4860 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0c98016000 tx=0x7f0c98017000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.441+0000 7f0ca8ba4700 1 -- 172.21.15.16:0/3852820252 reap_dead start 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.441+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 shutdown_connections 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.441+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 >> 172.21.15.16:0/3852820252 conn(0x7f0ca408a270 msgr2=0x7f0ca4153ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.444+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 --> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f68005c80 con 0x7f7f680015f0 2023-10-06T11:42:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.444+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 shutdown_connections 2023-10-06T11:42:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7fae12cb2700 1 -- 172.21.15.16:0/3444614613 wait complete. 2023-10-06T11:42:02.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 shutdown_connections 2023-10-06T11:42:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 >> 172.21.15.16:0/837572392 conn(0x7fd5ec06cee0 msgr2=0x7fd5ec0cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 shutdown_connections 2023-10-06T11:42:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7fd5f1445700 1 -- 172.21.15.16:0/837572392 wait complete. 2023-10-06T11:42:02.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7f7f7d7fa700 1 -- 172.21.15.16:0/19566943 <== osd.5 v2:172.21.15.177:6800/90843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f68005c80 con 0x7f7f680015f0 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f7f680015f0 msgr2=0x7f7f68003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7f7f8f586700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f7f680015f0 0x7f7f68003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.445+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f6c066f30 msgr2=0x7f7f6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.446+0000 7f7f8f586700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f6c066f30 0x7f7f6c0693e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009de0 tx=0x7f7f80019000 comp rx=0 tx=0).stop 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.446+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 msgr2=0x7f7f88158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.446+0000 7f7f8f586700 1 --2- 172.21.15.16:0/19566943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f88159800 0x7f7f88158e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f780010e0 tx=0x7f7f78007dd0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.447+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 shutdown_connections 2023-10-06T11:42:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.450+0000 7f7f8db23700 1 -- 172.21.15.16:0/19566943 reap_dead start 2023-10-06T11:42:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.450+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 shutdown_connections 2023-10-06T11:42:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.450+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 >> 172.21.15.16:0/19566943 conn(0x7f7f8808a230 msgr2=0x7f7f880b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.450+0000 7f0caa607700 1 -- 172.21.15.16:0/3852820252 wait complete. 2023-10-06T11:42:02.500 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:02.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:42:02.501 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:02.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:42:02.502 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:02.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:42:02.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.456+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 shutdown_connections 2023-10-06T11:42:02.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.456+0000 7f7f8f586700 1 -- 172.21.15.16:0/19566943 wait complete. 2023-10-06T11:42:02.504 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:02.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:42:02.505 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:02.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:42:02.506 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-10-06T11:42:02.506 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:42:02.507 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-10-06T11:42:02.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:42:02.508 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:02.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:42:02.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.741+0000 7fce1276e700 1 Processor -- start 2023-10-06T11:42:02.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.742+0000 7fce1276e700 1 -- start start 2023-10-06T11:42:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.743+0000 7fce1276e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c0c3a90 0x7fce0c15e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.743+0000 7fce1276e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c162920 con 0x7fce0c1475b0 2023-10-06T11:42:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.743+0000 7fce1276e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c06cbe0 con 0x7fce0c0c2f40 2023-10-06T11:42:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.743+0000 7fce1276e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c15e8a0 con 0x7fce0c0c3a90 2023-10-06T11:42:02.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce0c1475b0 0x7fce0c1619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56978/0 (socket says 172.21.15.16:56978) 2023-10-06T11:42:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0bfff700 1 -- 172.21.15.16:0/3297630910 learned_addr learned my addr 172.21.15.16:0/3297630910 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486583916 0 0) 0x7fce0c162920 con 0x7fce0c1475b0 2023-10-06T11:42:02.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf4003570 con 0x7fce0c1475b0 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519686056 0 0) 0x7fcdf4003570 con 0x7fce0c1475b0 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c0c3a90 msgr2=0x7fce0c15e360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c0c3a90 0x7fce0c15e360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 >> v1:172.21.15.177:6789/0 conn(0x7fce0c0c2f40 legacy=0x7fce0c0c3380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c1620d0 con 0x7fce0c1475b0 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.744+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fce040017f0 con 0x7fce0c1475b0 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 >> v1:172.21.15.16:6789/0 conn(0x7fce0c1475b0 legacy=0x7fce0c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcdf4003b10 0x7fcdf4003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdf4004810 0x7fcdf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf40053c0 0x7fcdf400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c15e8a0 con 0x7fcdf40053c0 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf400c2d0 con 0x7fcdf4004810 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf400c490 con 0x7fcdf4003b10 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.745+0000 7fce0bfff700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcdf4003b10 0x7fcdf4003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.746+0000 7fce0bfff700 1 -- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdf4004810 msgr2=0x7fcdf4004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.746+0000 7fce0bfff700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcdf4004810 0x7fcdf4004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.746+0000 7fce0bfff700 1 -- 172.21.15.16:0/3297630910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf40053c0 msgr2=0x7fcdf400ba60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:02.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.746+0000 7fce0bfff700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf40053c0 0x7fcdf400ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.746+0000 7fce0bfff700 1 -- 172.21.15.16:0/3297630910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c1620d0 con 0x7fcdf4003b10 2023-10-06T11:42:02.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.748+0000 7fb30d61a700 1 Processor -- start 2023-10-06T11:42:02.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.749+0000 7f51fd355700 1 Processor -- start 2023-10-06T11:42:02.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.749+0000 7f51fd355700 1 -- start start 2023-10-06T11:42:02.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f8849752700 1 Processor -- start 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.748+0000 7fb30d61a700 1 -- start start 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fb30d61a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813c1e0 0x7fb30806a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fb30d61a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30806a9a0 0x7fb3080c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fb30d61a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30815a860 con 0x7fb30813be10 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fb30d61a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30806ae40 con 0x7fb30806a9a0 2023-10-06T11:42:02.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fb30d61a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb308159a00 con 0x7fb30813c1e0 2023-10-06T11:42:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7fce0bfff700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcdf4003b10 0x7fcdf4003f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a930 tx=0x7fce0400ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7516a7fdf08549c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51fd355700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51f813c6d0 0x7f51f815b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51fd355700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f51f806a460 0x7f51f806a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51fd355700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f80c34e0 con 0x7f51f813caa0 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51fd355700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f806ae40 con 0x7f51f813c6d0 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51fd355700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f806c410 con 0x7f51f806a460 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51f77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f51f806a460 0x7f51f806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51f77fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f51f806a460 0x7f51f806a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45104/0 (socket says 172.21.15.16:45104) 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.750+0000 7f51f77fe700 1 -- 172.21.15.16:0/952767018 learned_addr learned my addr 172.21.15.16:0/952767018 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f6ffd700 1 --2- 172.21.15.16:0/952767018 >> v2:172.21.15.16:3300/0 conn(0x7f51f813c6d0 0x7f51f815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30806a9a0 0x7fb3080c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30806a9a0 0x7fb3080c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43148/0 (socket says 172.21.15.16:43148) 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 -- 172.21.15.16:0/2315091704 learned_addr learned my addr 172.21.15.16:0/2315091704 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 -- 172.21.15.16:0/2315091704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813c1e0 msgr2=0x7fb30806a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 --2- 172.21.15.16:0/2315091704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813c1e0 0x7fb30806a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 -- 172.21.15.16:0/2315091704 >> v1:172.21.15.177:6789/0 conn(0x7fb30813be10 legacy=0x7fb30806d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 -- 172.21.15.16:0/2315091704 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308159c50 con 0x7fb30806a9a0 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f77fe700 1 -- 172.21.15.16:0/952767018 >> v2:172.21.15.16:3300/0 conn(0x7f51f813c6d0 msgr2=0x7f51f815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f77fe700 1 --2- 172.21.15.16:0/952767018 >> v2:172.21.15.16:3300/0 conn(0x7f51f813c6d0 0x7f51f815b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f77fe700 1 -- 172.21.15.16:0/952767018 >> v1:172.21.15.16:6789/0 conn(0x7f51f813caa0 legacy=0x7f51f80c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f5ffb700 1 -- 172.21.15.16:0/952767018 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629736695 0 0) 0x7f51f80c34e0 con 0x7f51f813caa0 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f77fe700 1 -- 172.21.15.16:0/952767018 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f806c6f0 con 0x7f51f806a460 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f77fe700 1 --2- 172.21.15.16:0/952767018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f806a460 0x7f51f806a900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009f20 tx=0x7f51f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4f041b27f204a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7fb3077fe700 1 --2- 172.21.15.16:0/2315091704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30806a9a0 0x7fb3080c2ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb30000d1a0 tx=0x7fb30000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01836c46b09762f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f5ffb700 1 -- 172.21.15.16:0/952767018 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f000e070 con 0x7f51f806a460 2023-10-06T11:42:02.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.751+0000 7f51f5ffb700 1 -- 172.21.15.16:0/952767018 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f000ba40 con 0x7f51f806a460 2023-10-06T11:42:02.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51f5ffb700 1 -- 172.21.15.16:0/952767018 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00092e0 con 0x7f51f806a460 2023-10-06T11:42:02.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fb305ffb700 1 -- 172.21.15.16:0/2315091704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300010070 con 0x7fb30806a9a0 2023-10-06T11:42:02.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fb305ffb700 1 -- 172.21.15.16:0/2315091704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000dc70 con 0x7fb30806a9a0 2023-10-06T11:42:02.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fb305ffb700 1 -- 172.21.15.16:0/2315091704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300016440 con 0x7fb30806a9a0 2023-10-06T11:42:02.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04007110 con 0x7fcdf4003b10 2023-10-06T11:42:02.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04007270 con 0x7fcdf4003b10 2023-10-06T11:42:02.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7fce0affd700 1 -- 172.21.15.16:0/3297630910 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04007630 con 0x7fcdf4003b10 2023-10-06T11:42:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 -- 172.21.15.16:0/952767018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f806a460 msgr2=0x7f51f806a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 --2- 172.21.15.16:0/952767018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f806a460 0x7f51f806a900 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009f20 tx=0x7f51f0009840 comp rx=0 tx=0).stop 2023-10-06T11:42:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 -- 172.21.15.16:0/952767018 shutdown_connections 2023-10-06T11:42:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 --2- 172.21.15.16:0/952767018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f806a460 0x7f51f806a900 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 --2- 172.21.15.16:0/952767018 >> v2:172.21.15.16:3300/0 conn(0x7f51f813c6d0 0x7f51f815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.752+0000 7f51fd355700 1 -- 172.21.15.16:0/952767018 >> 172.21.15.16:0/952767018 conn(0x7f51f8068350 msgr2=0x7f51f8068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 -- 172.21.15.16:0/2315091704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30806a9a0 msgr2=0x7fb3080c2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 --2- 172.21.15.16:0/2315091704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30806a9a0 0x7fb3080c2ae0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb30000d1a0 tx=0x7fb30000ba50 comp rx=0 tx=0).stop 2023-10-06T11:42:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 -- 172.21.15.16:0/2315091704 shutdown_connections 2023-10-06T11:42:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 --2- 172.21.15.16:0/2315091704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30806a9a0 0x7fb3080c2ae0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 --2- 172.21.15.16:0/2315091704 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813c1e0 0x7fb30806a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 -- 172.21.15.16:0/2315091704 >> 172.21.15.16:0/2315091704 conn(0x7fb308068350 msgr2=0x7fb308068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 -- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcdf4003b10 msgr2=0x7fcdf4003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 --2- 172.21.15.16:0/3297630910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcdf4003b10 0x7fcdf4003f00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a930 tx=0x7fce0400ac40 comp rx=0 tx=0).stop 2023-10-06T11:42:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce10d0b700 1 -- 172.21.15.16:0/3297630910 reap_dead start 2023-10-06T11:42:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 -- 172.21.15.16:0/3297630910 shutdown_connections 2023-10-06T11:42:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 -- 172.21.15.16:0/3297630910 >> 172.21.15.16:0/3297630910 conn(0x7fce0c068350 msgr2=0x7fce0c068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7f51fd355700 1 -- 172.21.15.16:0/952767018 shutdown_connections 2023-10-06T11:42:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 -- 172.21.15.16:0/2315091704 shutdown_connections 2023-10-06T11:42:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fb30d61a700 1 -- 172.21.15.16:0/2315091704 wait complete. 2023-10-06T11:42:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 -- 172.21.15.16:0/3297630910 shutdown_connections 2023-10-06T11:42:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7fce1276e700 1 -- 172.21.15.16:0/3297630910 wait complete. 2023-10-06T11:42:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.754+0000 7fb30d61a700 1 Processor -- start 2023-10-06T11:42:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fce1276e700 1 Processor -- start 2023-10-06T11:42:02.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.753+0000 7f51fd355700 1 -- 172.21.15.16:0/952767018 wait complete. 2023-10-06T11:42:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7f8849752700 1 -- start start 2023-10-06T11:42:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 -- start start 2023-10-06T11:42:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb30806a9a0 0x7fb308150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813be10 0x7fb308154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb308150750 con 0x7fb30813c1e0 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c5ee0 con 0x7fb30806a9a0 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fb30d61a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3080c8f90 con 0x7fb30813be10 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43170/0 (socket says 172.21.15.16:43170) 2023-10-06T11:42:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 -- 172.21.15.16:0/1083196831 learned_addr learned my addr 172.21.15.16:0/1083196831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7f51fd355700 1 Processor -- start 2023-10-06T11:42:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7f51fd355700 1 -- start start 2023-10-06T11:42:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.755+0000 7fce1276e700 1 -- start start 2023-10-06T11:42:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 -- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813be10 msgr2=0x7fb308154020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813be10 0x7fb308154020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 -- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb30806a9a0 msgr2=0x7fb308150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb30806a9a0 0x7fb308150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.756+0000 7fb3077fe700 1 -- 172.21.15.16:0/1083196831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30000bcf0 con 0x7fb30813c1e0 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb3077fe700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ba90 tx=0x7fb30001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f8849752700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88440ce1e0 0x7f884406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f8849752700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8844159240 0x7f8844159680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f8849752700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88440c88e0 con 0x7f884406c5d0 2023-10-06T11:42:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f8849752700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88440c8d70 con 0x7f8844159240 2023-10-06T11:42:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f8849752700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8844159bc0 con 0x7f88440ce1e0 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51f806a460 0x7f51f8178dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f813c6d0 0x7f51f816a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f815c2d0 con 0x7f51f813caa0 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f815b6c0 con 0x7f51f813c6d0 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51fd355700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f815b9a0 con 0x7f51f806a460 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43200/0 (socket says 172.21.15.16:43200) 2023-10-06T11:42:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 -- 172.21.15.16:0/324949537 learned_addr learned my addr 172.21.15.16:0/324949537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 0x7fce0c16f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c147980 0x7fce0c0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30000b370 con 0x7fb30813c1e0 2023-10-06T11:42:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30000b4d0 con 0x7fb30813c1e0 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 0x7fce0c1436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0c8480 con 0x7fcdf4003b10 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c143c20 con 0x7fce0c143270 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce1276e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c16f900 con 0x7fce0c147980 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce0bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 0x7fce0c16f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300023950 con 0x7fb30813c1e0 2023-10-06T11:42:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce0bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 0x7fce0c16f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43186/0 (socket says 172.21.15.16:43186) 2023-10-06T11:42:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce0bfff700 1 -- 172.21.15.16:0/1925800604 learned_addr learned my addr 172.21.15.16:0/1925800604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fce10d0b700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 0x7fce0c1436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f6ffd700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51f806a460 0x7f51f8178dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3080c89d0 con 0x7fb30813c1e0 2023-10-06T11:42:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3080c5b70 con 0x7fb30813c1e0 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 -- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51f806a460 msgr2=0x7f51f8178dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51f806a460 0x7f51f8178dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 -- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f813c6d0 msgr2=0x7f51f816a4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f813c6d0 0x7f51f816a4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f51f77fe700 1 -- 172.21.15.16:0/324949537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0009ae0 con 0x7f51f813caa0 2023-10-06T11:42:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f51f77fe700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b6c0 tx=0x7f51f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fce10d0b700 1 -- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c147980 msgr2=0x7fce0c0c7a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fce10d0b700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c147980 0x7fce0c0c7a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fce10d0b700 1 -- 172.21.15.16:0/1925800604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 msgr2=0x7fce0c16f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fce10d0b700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 0x7fce0c16f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fce10d0b700 1 -- 172.21.15.16:0/1925800604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c16f590 con 0x7fce0c143270 2023-10-06T11:42:02.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f88437fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8844159240 0x7f8844159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f88437fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8844159240 0x7f8844159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43188/0 (socket says 172.21.15.16:43188) 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.757+0000 7f88437fe700 1 -- 172.21.15.16:0/2175458630 learned_addr learned my addr 172.21.15.16:0/2175458630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f88437fe700 1 -- 172.21.15.16:0/2175458630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88440ce1e0 msgr2=0x7f884406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f88437fe700 1 --2- 172.21.15.16:0/2175458630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88440ce1e0 0x7f884406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f88437fe700 1 -- 172.21.15.16:0/2175458630 >> v1:172.21.15.177:6789/0 conn(0x7f884406c5d0 legacy=0x7f88440cb790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f88437fe700 1 -- 172.21.15.16:0/2175458630 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440c7cd0 con 0x7f8844159240 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7f88437fe700 1 --2- 172.21.15.16:0/2175458630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8844159240 0x7f8844159680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f883c009f20 tx=0x7f883c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de254ccacd178809 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fea31652700 1 Processor -- start 2023-10-06T11:42:02.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb308063e70 con 0x7fb30813c1e0 2023-10-06T11:42:02.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f8841ffb700 1 -- 172.21.15.16:0/2175458630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00e070 con 0x7f8844159240 2023-10-06T11:42:02.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f8841ffb700 1 -- 172.21.15.16:0/2175458630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c00ba40 con 0x7f8844159240 2023-10-06T11:42:02.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f8841ffb700 1 -- 172.21.15.16:0/2175458630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0092e0 con 0x7f8844159240 2023-10-06T11:42:02.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7fce10d0b700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 0x7fce0c1436e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40057f0 tx=0x7fcdfc006ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.758+0000 7fea31652700 1 -- start start 2023-10-06T11:42:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0008e20 con 0x7f51f813caa0 2023-10-06T11:42:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f000be50 con 0x7f51f813caa0 2023-10-06T11:42:02.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00266b0 con 0x7f51f813caa0 2023-10-06T11:42:02.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7ff394b21700 1 Processor -- start 2023-10-06T11:42:02.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea31652700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea2c1558f0 0x7fea2c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f8849752700 1 -- 172.21.15.16:0/2175458630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8844159240 msgr2=0x7f8844159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f8849752700 1 --2- 172.21.15.16:0/2175458630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8844159240 0x7f8844159680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f883c009f20 tx=0x7f883c009840 comp rx=0 tx=0).stop 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea31652700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c06af40 con 0x7fea2c156e30 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea31652700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c06a610 con 0x7fea2c06b3d0 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea31652700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c06a8f0 con 0x7fea2c1558f0 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f4a66cab700 1 Processor -- start 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2affd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fea2c1558f0 0x7fea2c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2b7fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fea2c06b3d0 0x7fea2c0c6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48750/0 (socket says 172.21.15.16:48750) 2023-10-06T11:42:02.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2b7fe700 1 -- 172.21.15.16:0/1711357947 learned_addr learned my addr 172.21.15.16:0/1711357947 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:02.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f815bc80 con 0x7f51f813caa0 2023-10-06T11:42:02.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f4a66cab700 1 -- start start 2023-10-06T11:42:02.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8179310 con 0x7f51f813caa0 2023-10-06T11:42:02.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f351b040700 1 Processor -- start 2023-10-06T11:42:02.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2affd700 1 -- 172.21.15.16:0/1711357947 >> v1:172.21.15.177:6789/0 conn(0x7fea2c06b3d0 legacy=0x7fea2c0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2affd700 1 -- 172.21.15.16:0/1711357947 >> v1:172.21.15.16:6789/0 conn(0x7fea2c156e30 legacy=0x7fea2c0c8b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:02.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fea2affd700 1 -- 172.21.15.16:0/1711357947 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c06abd0 con 0x7fea2c1558f0 2023-10-06T11:42:02.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7f351b040700 1 -- start start 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.759+0000 7ff394b21700 1 -- start start 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7ff394b21700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff39013be10 0x7ff39006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7ff394b21700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39015a860 con 0x7ff39013c1e0 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7ff394b21700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff390159a00 con 0x7ff39006ab70 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7ff394b21700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff390159c50 con 0x7ff39013be10 2023-10-06T11:42:02.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fea2affd700 1 --2- 172.21.15.16:0/1711357947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c1558f0 0x7fea2c1568f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00aa90 tx=0x7fea1c00a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e722c8c81cd02109 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f8849752700 1 -- 172.21.15.16:0/2175458630 shutdown_connections 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f8849752700 1 --2- 172.21.15.16:0/2175458630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8844159240 0x7f8844159680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f8849752700 1 --2- 172.21.15.16:0/2175458630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88440ce1e0 0x7f884406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f8849752700 1 -- 172.21.15.16:0/2175458630 >> 172.21.15.16:0/2175458630 conn(0x7f884406d520 msgr2=0x7f884406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff39013be10 0x7ff39006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff39013be10 0x7ff39006d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45176/0 (socket says 172.21.15.16:45176) 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 -- 172.21.15.16:0/340139788 learned_addr learned my addr 172.21.15.16:0/340139788 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fea29ffb700 1 -- 172.21.15.16:0/1711357947 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00ca00 con 0x7fea2c1558f0 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fea29ffb700 1 -- 172.21.15.16:0/1711357947 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00cb60 con 0x7fea2c1558f0 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fea29ffb700 1 -- 172.21.15.16:0/1711357947 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0183f0 con 0x7fea2c1558f0 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.760+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb300023d40 con 0x7fb30813c1e0 2023-10-06T11:42:02.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fb2eeffd700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 0x7fb2f4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb30008a9a0 con 0x7fb30813c1e0 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f4a66cab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f4a66cab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a60162920 con 0x7f4a60147980 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f4a66cab700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6006cbe0 con 0x7f4a600c2f40 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f4a66cab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6015e8a0 con 0x7f4a601475b0 2023-10-06T11:42:02.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0038a0 con 0x7fce0c143270 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc003a00 con 0x7fce0c143270 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003d80 con 0x7fce0c143270 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c0c7f40 con 0x7fce0c143270 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fb306ffd700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 0x7fb2f4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f351b040700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351406e660 0x7f3514099d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f351b040700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7f351b040700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3514098c80 con 0x7f3514161890 2023-10-06T11:42:02.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f351b040700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351415e200 con 0x7f351406e660 2023-10-06T11:42:02.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f351b040700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3514098f60 con 0x7f3514098780 2023-10-06T11:42:02.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c148290 con 0x7fce0c143270 2023-10-06T11:42:02.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f815c440 con 0x7f51f813caa0 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 -- 172.21.15.16:0/340139788 >> v1:172.21.15.177:6789/0 conn(0x7ff39006ab70 legacy=0x7ff3900c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 -- 172.21.15.16:0/340139788 >> v1:172.21.15.16:6789/0 conn(0x7ff39013c1e0 legacy=0x7ff39006a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 -- 172.21.15.16:0/340139788 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390159f30 con 0x7ff39013be10 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.761+0000 7ff38e59c700 1 --2- 172.21.15.16:0/340139788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39013be10 0x7ff39006d4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3800099b0 tx=0x7ff380009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e360a1342c6f2b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7ff38d59a700 1 -- 172.21.15.16:0/340139788 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000b820 con 0x7ff39013be10 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7ff38d59a700 1 -- 172.21.15.16:0/340139788 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000b980 con 0x7ff39013be10 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7ff38d59a700 1 -- 172.21.15.16:0/340139788 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000bd20 con 0x7ff39013be10 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41708/0 (socket says 172.21.15.16:41708) 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 -- 172.21.15.16:0/1219662180 learned_addr learned my addr 172.21.15.16:0/1219662180 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3518ddc700 1 --2- 172.21.15.16:0/1219662180 >> v2:172.21.15.16:3300/0 conn(0x7f351406e660 0x7f3514099d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fea31652700 1 -- 172.21.15.16:0/1711357947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c1558f0 msgr2=0x7fea2c1568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fea31652700 1 --2- 172.21.15.16:0/1711357947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c1558f0 0x7fea2c1568f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00aa90 tx=0x7fea1c00a330 comp rx=0 tx=0).stop 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41698/0 (socket says 172.21.15.16:41698) 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 -- 172.21.15.16:0/2826325007 learned_addr learned my addr 172.21.15.16:0/2826325007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 -- 172.21.15.16:0/2826325007 >> v1:172.21.15.177:6789/0 conn(0x7f4a600c2f40 legacy=0x7f4a601619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 -- 172.21.15.16:0/2826325007 >> v1:172.21.15.16:6789/0 conn(0x7f4a60147980 legacy=0x7f4a600c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 -- 172.21.15.16:0/2826325007 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a601620d0 con 0x7f4a601475b0 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 -- 172.21.15.16:0/1219662180 >> v2:172.21.15.16:3300/0 conn(0x7f351406e660 msgr2=0x7f3514099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 --2- 172.21.15.16:0/1219662180 >> v2:172.21.15.16:3300/0 conn(0x7f351406e660 0x7f3514099d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 -- 172.21.15.16:0/1219662180 >> v1:172.21.15.16:6789/0 conn(0x7f3514161890 legacy=0x7f35140ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f3513fff700 1 -- 172.21.15.16:0/1219662180 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514099240 con 0x7f3514098780 2023-10-06T11:42:02.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fea31652700 1 -- 172.21.15.16:0/1711357947 shutdown_connections 2023-10-06T11:42:02.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fea31652700 1 --2- 172.21.15.16:0/1711357947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c1558f0 0x7fea2c1568f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7fea31652700 1 -- 172.21.15.16:0/1711357947 >> 172.21.15.16:0/1711357947 conn(0x7fea2c06c410 msgr2=0x7fea2c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.762+0000 7f4a64a47700 1 --2- 172.21.15.16:0/2826325007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099e0 tx=0x7f4a50009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59207fa4e4b7b6b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7fea31652700 1 -- 172.21.15.16:0/1711357947 shutdown_connections 2023-10-06T11:42:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7f3513fff700 1 --2- 172.21.15.16:0/1219662180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35080099b0 tx=0x7f3508009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3465bf58b65493c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7fea31652700 1 -- 172.21.15.16:0/1711357947 wait complete. 2023-10-06T11:42:02.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7f4a5f7fe700 1 -- 172.21.15.16:0/2826325007 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000b820 con 0x7f4a601475b0 2023-10-06T11:42:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7fb306ffd700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 0x7fb2f4069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097c0 tx=0x7fb2f8006ce0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7f4a5f7fe700 1 -- 172.21.15.16:0/2826325007 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5000b980 con 0x7f4a601475b0 2023-10-06T11:42:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7f4a5f7fe700 1 -- 172.21.15.16:0/2826325007 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000bd20 con 0x7f4a601475b0 2023-10-06T11:42:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c16fa70 con 0x7fce0c143270 2023-10-06T11:42:02.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7ff394b21700 1 -- 172.21.15.16:0/340139788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39013be10 msgr2=0x7ff39006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.763+0000 7ff394b21700 1 --2- 172.21.15.16:0/340139788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39013be10 0x7ff39006d4e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3800099b0 tx=0x7ff380009250 comp rx=0 tx=0).stop 2023-10-06T11:42:02.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f8849752700 1 -- 172.21.15.16:0/2175458630 shutdown_connections 2023-10-06T11:42:02.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7ff394b21700 1 -- 172.21.15.16:0/340139788 shutdown_connections 2023-10-06T11:42:02.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7ff394b21700 1 --2- 172.21.15.16:0/340139788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39013be10 0x7ff39006d4e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7ff394b21700 1 -- 172.21.15.16:0/340139788 >> 172.21.15.16:0/340139788 conn(0x7ff390068350 msgr2=0x7ff390068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f8849752700 1 -- 172.21.15.16:0/2175458630 wait complete. 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f35137fe700 1 -- 172.21.15.16:0/1219662180 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800b860 con 0x7f3514098780 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f35137fe700 1 -- 172.21.15.16:0/1219662180 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800b9c0 con 0x7f3514098780 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f35137fe700 1 -- 172.21.15.16:0/1219662180 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800bd60 con 0x7f3514098780 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f351b040700 1 -- 172.21.15.16:0/1219662180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 msgr2=0x7f351406ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.764+0000 7f351b040700 1 --2- 172.21.15.16:0/1219662180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35080099b0 tx=0x7f3508009250 comp rx=0 tx=0).stop 2023-10-06T11:42:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7fea31652700 1 Processor -- start 2023-10-06T11:42:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7ff394b21700 1 -- 172.21.15.16:0/340139788 shutdown_connections 2023-10-06T11:42:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7ff394b21700 1 -- 172.21.15.16:0/340139788 wait complete. 2023-10-06T11:42:02.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 -- 172.21.15.16:0/1219662180 shutdown_connections 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 --2- 172.21.15.16:0/1219662180 >> v2:172.21.15.16:3300/0 conn(0x7f351406e660 0x7f3514099d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 --2- 172.21.15.16:0/1219662180 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351406ea30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 -- 172.21.15.16:0/1219662180 >> 172.21.15.16:0/1219662180 conn(0x7f35140d1250 msgr2=0x7f35140cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7fea31652700 1 -- start start 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 -- 172.21.15.16:0/1219662180 shutdown_connections 2023-10-06T11:42:02.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f351b040700 1 -- 172.21.15.16:0/1219662180 wait complete. 2023-10-06T11:42:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f8849752700 1 Processor -- start 2023-10-06T11:42:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.765+0000 7f8849752700 1 -- start start 2023-10-06T11:42:02.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 -- 172.21.15.16:0/2826325007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 msgr2=0x7f4a6015e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 --2- 172.21.15.16:0/2826325007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099e0 tx=0x7f4a50009250 comp rx=0 tx=0).stop 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 -- 172.21.15.16:0/2826325007 shutdown_connections 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 --2- 172.21.15.16:0/2826325007 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a6015e360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 -- 172.21.15.16:0/2826325007 >> 172.21.15.16:0/2826325007 conn(0x7f4a60068350 msgr2=0x7f4a60068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 -- 172.21.15.16:0/2826325007 shutdown_connections 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7ff394b21700 1 Processor -- start 2023-10-06T11:42:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea2c1558f0 0x7fea2c147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 0x7fea2c0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0c30c0 con 0x7fea2c06b3d0 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c168af0 con 0x7fea2c156e30 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea31652700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c147580 con 0x7fea2c1558f0 2023-10-06T11:42:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7ff394b21700 1 -- start start 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43226/0 (socket says 172.21.15.16:43226) 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2affd700 1 -- 172.21.15.16:0/3610117668 learned_addr learned my addr 172.21.15.16:0/3610117668 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 0x7fea2c0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 -- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea2c1558f0 msgr2=0x7fea2c147040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea2c1558f0 0x7fea2c147040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 -- 172.21.15.16:0/3610117668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 msgr2=0x7fea2c1685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2b7fe700 1 -- 172.21.15.16:0/3610117668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c00a5d0 con 0x7fea2c156e30 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7fea2affd700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 0x7f8844159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 0x7f884414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88440ae3d0 con 0x7f884406c5d0 2023-10-06T11:42:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f351b040700 1 Processor -- start 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884415d010 con 0x7f88440ce1e0 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f8849752700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884415cd30 con 0x7f8844159240 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88427fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88427fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45204/0 (socket says 172.21.15.16:45204) 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88427fc700 1 -- 172.21.15.16:0/3670126329 learned_addr learned my addr 172.21.15.16:0/3670126329 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88437fe700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 0x7f884414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 0x7f8844159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f351b040700 1 -- start start 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 -- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 msgr2=0x7f884414cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 0x7f884414cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 -- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 msgr2=0x7f88440b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 -- 172.21.15.16:0/3670126329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c009ae0 con 0x7f884406c5d0 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88427fc700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f88437fe700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 0x7f884414cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f8842ffd700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 0x7f8844159dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f883800b530 tx=0x7f88380090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 0x7ff390158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39013c1e0 0x7ff390153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390117e20 con 0x7ff39013be10 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901180c0 con 0x7ff39006ab70 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff394b21700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901183a0 con 0x7ff39013c1e0 2023-10-06T11:42:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff38dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.766+0000 7f4a66cab700 1 -- 172.21.15.16:0/2826325007 wait complete. 2023-10-06T11:42:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7f4a66cab700 1 Processor -- start 2023-10-06T11:42:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.767+0000 7ff38dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43252/0 (socket says 172.21.15.16:43252) 2023-10-06T11:42:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 -- 172.21.15.16:0/1271051885 learned_addr learned my addr 172.21.15.16:0/1271051885 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f4a66cab700 1 -- start start 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800f510 con 0x7f884406c5d0 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88440add80 con 0x7f884406c5d0 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883800f690 con 0x7f884406c5d0 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800fa50 con 0x7f884406c5d0 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884415c970 con 0x7f884406c5d0 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 -- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39013c1e0 msgr2=0x7ff390153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39013c1e0 0x7ff390153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 -- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 msgr2=0x7ff390158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38e59c700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 0x7ff390158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 0x7ff390158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38dd9b700 1 -- 172.21.15.16:0/1271051885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800094f0 con 0x7ff39013be10 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7ff38e59c700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 0x7ff390158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb300057b10 con 0x7fb30813c1e0 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a600c2f40 0x7f4a60171ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60147980 0x7f4a6016f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a600c6950 con 0x7f4a60147980 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a600c71f0 con 0x7f4a600c2f40 2023-10-06T11:42:02.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a66cab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a60141990 con 0x7f4a601475b0 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f351406e660 0x7f35140b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3514161890 0x7f3514171820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b4aa0 con 0x7f3514161890 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b4d50 con 0x7f351406e660 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f351b040700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b5030 con 0x7f3514098780 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f3513fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f3513fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41726/0 (socket says 172.21.15.16:41726) 2023-10-06T11:42:02.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f3513fff700 1 -- 172.21.15.16:0/3720281967 learned_addr learned my addr 172.21.15.16:0/3720281967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f35195dd700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3514161890 0x7f3514171820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.768+0000 7f3518ddc700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f351406e660 0x7f35140b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 -- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f351406e660 msgr2=0x7f35140b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f351406e660 0x7f35140b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 -- 172.21.15.16:0/3720281967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3514161890 msgr2=0x7f3514171820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3514161890 0x7f3514171820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 -- 172.21.15.16:0/3720281967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080094f0 con 0x7f3514098780 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3513fff700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35140990a0 tx=0x7f3508017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8844063e70 con 0x7f884406c5d0 2023-10-06T11:42:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a5ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a5ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41728/0 (socket says 172.21.15.16:41728) 2023-10-06T11:42:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a5ffff700 1 -- 172.21.15.16:0/4265034217 learned_addr learned my addr 172.21.15.16:0/4265034217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508011d30 con 0x7f3514098780 2023-10-06T11:42:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a64a47700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a600c2f40 0x7f4a60171ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508011e90 con 0x7f3514098780 2023-10-06T11:42:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35140b5310 con 0x7f3514098780 2023-10-06T11:42:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f4a65248700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60147980 0x7f4a6016f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.769+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35140b58d0 con 0x7f3514098780 2023-10-06T11:42:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508024690 con 0x7f3514098780 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 -- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a600c2f40 msgr2=0x7f4a60171ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a600c2f40 0x7f4a60171ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 -- 172.21.15.16:0/4265034217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60147980 msgr2=0x7f4a6016f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60147980 0x7f4a6016f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 -- 172.21.15.16:0/4265034217 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500094f0 con 0x7f4a601475b0 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7f4a5ffff700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c910 tx=0x7f4a58002720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58012630 con 0x7f4a601475b0 2023-10-06T11:42:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a580127b0 con 0x7f4a601475b0 2023-10-06T11:42:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58012b70 con 0x7f4a601475b0 2023-10-06T11:42:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35140b5ac0 con 0x7f3514098780 2023-10-06T11:42:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f350802d440 con 0x7f3514098780 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.770+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdfc0063a0 con 0x7fce0c143270 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7fcdf3fff700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 0x7fcdf8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcdfc0837c0 con 0x7fce0c143270 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7fce0bfff700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 0x7fcdf8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdfc085d60 con 0x7fce0c143270 2023-10-06T11:42:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a600c6ba0 con 0x7f4a601475b0 2023-10-06T11:42:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7fce0bfff700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 0x7fcdf8069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a450 tx=0x7fce04015000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7f3510ff9700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 0x7f35000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3508086060 con 0x7f3514098780 2023-10-06T11:42:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7f3518ddc700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 0x7f35000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.771+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601414c0 con 0x7f4a601475b0 2023-10-06T11:42:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a600c7360 con 0x7f4a601475b0 2023-10-06T11:42:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7f3518ddc700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 0x7f35000693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.772+0000 7fea2b7fe700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 0x7fea2c0c5680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c830 tx=0x7fea240052a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.773+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24012630 con 0x7fea2c156e30 2023-10-06T11:42:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.773+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea240127b0 con 0x7fea2c156e30 2023-10-06T11:42:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.773+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c0c24b0 con 0x7fea2c156e30 2023-10-06T11:42:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.773+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c0c2a70 con 0x7fea2c156e30 2023-10-06T11:42:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.773+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24012b70 con 0x7fea2c156e30 2023-10-06T11:42:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51f0003040 con 0x7f51f813caa0 2023-10-06T11:42:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7ff38dd9b700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c4e0 tx=0x7ff384007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7f51deffd700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 0x7f51e4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0c0052b0 con 0x7fea2c156e30 2023-10-06T11:42:02.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51f0086890 con 0x7f51f813caa0 2023-10-06T11:42:02.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7f51f6ffd700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 0x7f51e4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.774+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea2401b400 con 0x7fea2c156e30 2023-10-06T11:42:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.775+0000 7f51f6ffd700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 0x7f51e4069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.775+0000 7fea12ffd700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 0x7fea180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.775+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea24084500 con 0x7fea2c156e30 2023-10-06T11:42:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.775+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51f0053ab0 con 0x7f51f813caa0 2023-10-06T11:42:02.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.775+0000 7fea2affd700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 0x7fea180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.776+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384011590 con 0x7ff39013be10 2023-10-06T11:42:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.776+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff390118680 con 0x7ff39013be10 2023-10-06T11:42:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.776+0000 7fea2affd700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 0x7fea180693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00a5a0 tx=0x7fea1c019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.776+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff390118c40 con 0x7ff39013be10 2023-10-06T11:42:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.776+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8838024430 con 0x7f884406c5d0 2023-10-06T11:42:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.777+0000 7f882affd700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 0x7f88300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.777+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8838084950 con 0x7f884406c5d0 2023-10-06T11:42:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.777+0000 7f88427fc700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 0x7f88300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.777+0000 7f88427fc700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 0x7f88300693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f884415c730 tx=0x7f8834009250 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.778+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff384011900 con 0x7ff39013be10 2023-10-06T11:42:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.778+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff384011a70 con 0x7ff39013be10 2023-10-06T11:42:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.779+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3508053150 con 0x7f3514098780 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.779+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a5801b400 con 0x7f4a601475b0 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.780+0000 7f4a5cff9700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 0x7f4a4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.780+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a58084ff0 con 0x7f4a601475b0 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.780+0000 7f4a64a47700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 0x7f4a4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.781+0000 7f4a64a47700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 0x7f4a4c0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099e0 tx=0x7f4a50016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.783+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea24051610 con 0x7fea2c156e30 2023-10-06T11:42:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.784+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3740052b0 con 0x7ff39013be10 2023-10-06T11:42:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.784+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8838051bc0 con 0x7f884406c5d0 2023-10-06T11:42:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.785+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a58052120 con 0x7f4a601475b0 2023-10-06T11:42:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.790+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff38401c440 con 0x7ff39013be10 2023-10-06T11:42:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.791+0000 7ff37e7fc700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.791+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff384084370 con 0x7ff39013be10 2023-10-06T11:42:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.794+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff384051490 con 0x7ff39013be10 2023-10-06T11:42:02.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.802+0000 7ff38e59c700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:02.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:02.845+0000 7ff38e59c700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 0x7ff3700693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3800099b0 tx=0x7ff380016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:03.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.166+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb308159f30 con 0x7fb30813c1e0 2023-10-06T11:42:03.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.167+0000 7fb2eeffd700 1 -- 172.21.15.16:0/1083196831 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb30005b130 con 0x7fb30813c1e0 2023-10-06T11:42:03.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.168+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51f8063e70 con 0x7f51f813caa0 2023-10-06T11:42:03.170 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:03.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.168+0000 7f51deffd700 1 -- 172.21.15.16:0/324949537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51f0021020 con 0x7f51f813caa0 2023-10-06T11:42:03.172 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 msgr2=0x7fb2f4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 0x7fb2f4069530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2f80097c0 tx=0x7fb2f8006ce0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 msgr2=0x7fb308154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb30000ba90 tx=0x7fb30001b000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 shutdown_connections 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30813c1e0 0x7fb308154560 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb2f4067080 0x7fb2f4069530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb30806a9a0 0x7fb308150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 --2- 172.21.15.16:0/1083196831 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb30813be10 0x7fb308154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 >> 172.21.15.16:0/1083196831 conn(0x7fb308068350 msgr2=0x7fb3080c2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 shutdown_connections 2023-10-06T11:42:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.171+0000 7fb30d61a700 1 -- 172.21.15.16:0/1083196831 wait complete. 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 msgr2=0x7f51e4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 0x7f51e4069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 msgr2=0x7f51f8175fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f51f000b6c0 tx=0x7f51f0018000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 shutdown_connections 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f813caa0 0x7f51f8175fa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f51e4067080 0x7f51e4069530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51f806a460 0x7f51f8178dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 --2- 172.21.15.16:0/324949537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51f813c6d0 0x7f51f816a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.172+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 >> 172.21.15.16:0/324949537 conn(0x7f51f8068350 msgr2=0x7f51f813f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.173+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 shutdown_connections 2023-10-06T11:42:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.173+0000 7f51fd355700 1 -- 172.21.15.16:0/324949537 wait complete. 2023-10-06T11:42:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.173+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce0c063e70 con 0x7fce0c143270 2023-10-06T11:42:03.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.175+0000 7fcdf3fff700 1 -- 172.21.15.16:0/1925800604 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdfc050930 con 0x7fce0c143270 2023-10-06T11:42:03.180 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:03.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 msgr2=0x7fcdf8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 0x7fcdf8069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a450 tx=0x7fce04015000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 msgr2=0x7fce0c1436e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 0x7fce0c1436e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40057f0 tx=0x7fcdfc006ee0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f4a60063e70 con 0x7f4a601475b0 2023-10-06T11:42:03.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 shutdown_connections 2023-10-06T11:42:03.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdf4003b10 0x7fce0c16f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcdf8067080 0x7fcdf8069530 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fce0400a450 tx=0x7fce04015000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce0c143270 0x7fce0c1436e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 --2- 172.21.15.16:0/1925800604 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce0c147980 0x7fce0c0c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.188+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 >> 172.21.15.16:0/1925800604 conn(0x7fce0c068350 msgr2=0x7fce0c15e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.189+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 shutdown_connections 2023-10-06T11:42:03.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.189+0000 7fce1276e700 1 -- 172.21.15.16:0/1925800604 wait complete. 2023-10-06T11:42:03.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.191+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f35140b5170 con 0x7f3514098780 2023-10-06T11:42:03.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.191+0000 7f3510ff9700 1 -- 172.21.15.16:0/3720281967 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35140b5170 con 0x7f3514098780 2023-10-06T11:42:03.194 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:03.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-06T11:42:03.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2023-10-06T11:42:03.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.194+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 msgr2=0x7f35000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.194+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 0x7f35000693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3504006fd0 tx=0x7f3504008040 comp rx=0 tx=0).stop 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.194+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 msgr2=0x7f351415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.194+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f35140990a0 tx=0x7f3508017000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 shutdown_connections 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3514161890 0x7f3514171820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3500066f30 0x7f35000693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f351406e660 0x7f35140b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 --2- 172.21.15.16:0/3720281967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3514098780 0x7f351415e500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 >> 172.21.15.16:0/3720281967 conn(0x7f35140d1250 msgr2=0x7f351415d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 shutdown_connections 2023-10-06T11:42:03.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.195+0000 7f351b040700 1 -- 172.21.15.16:0/3720281967 wait complete. 2023-10-06T11:42:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.197+0000 7f4a5cff9700 1 -- 172.21.15.16:0/4265034217 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a58016020 con 0x7f4a601475b0 2023-10-06T11:42:03.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.197+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff374005830 con 0x7ff39013be10 2023-10-06T11:42:03.202 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:03.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.197+0000 7ff37e7fc700 1 -- 172.21.15.16:0/1271051885 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff384017020 con 0x7ff39013be10 2023-10-06T11:42:03.203 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:03.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 msgr2=0x7ff3700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 0x7ff3700693e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3800099b0 tx=0x7ff380016000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 msgr2=0x7ff390153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c4e0 tx=0x7ff384007f00 comp rx=0 tx=0).stop 2023-10-06T11:42:03.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 msgr2=0x7f4a4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 0x7f4a4c0693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a500099e0 tx=0x7f4a50016000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 msgr2=0x7f4a60140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.200+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c910 tx=0x7f4a58002720 comp rx=0 tx=0).stop 2023-10-06T11:42:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 shutdown_connections 2023-10-06T11:42:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff370066f30 0x7ff3700693e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff39013c1e0 0x7ff390153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff39006ab70 0x7ff390158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 --2- 172.21.15.16:0/1271051885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff39013be10 0x7ff390153110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 >> 172.21.15.16:0/1271051885 conn(0x7ff390068350 msgr2=0x7ff3900cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 shutdown_connections 2023-10-06T11:42:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.201+0000 7ff394b21700 1 -- 172.21.15.16:0/1271051885 wait complete. 2023-10-06T11:42:03.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 shutdown_connections 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a60147980 0x7f4a6016f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a4c066f30 0x7f4a4c0693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a600c2f40 0x7f4a60171ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 --2- 172.21.15.16:0/4265034217 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a601475b0 0x7f4a60140f80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 >> 172.21.15.16:0/4265034217 conn(0x7f4a60068350 msgr2=0x7f4a60160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 shutdown_connections 2023-10-06T11:42:03.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.202+0000 7f4a66cab700 1 -- 172.21.15.16:0/4265034217 wait complete. 2023-10-06T11:42:03.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-06T11:42:03.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-06T11:42:03.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-06T11:42:03.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-06T11:42:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.224+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88440c7fb0 con 0x7f884406c5d0 2023-10-06T11:42:03.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.223+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fea0c005830 con 0x7fea2c156e30 2023-10-06T11:42:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.224+0000 7fea12ffd700 1 -- 172.21.15.16:0/3610117668 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fea24054c30 con 0x7fea2c156e30 2023-10-06T11:42:03.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.224+0000 7f882affd700 1 -- 172.21.15.16:0/3670126329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88380551e0 con 0x7f884406c5d0 2023-10-06T11:42:03.227 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:03.228 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 msgr2=0x7fea180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 0x7fea180693e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00a5a0 tx=0x7fea1c019000 comp rx=0 tx=0).stop 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 msgr2=0x7fea2c0c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 0x7fea2c0c5680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c830 tx=0x7fea240052a0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 shutdown_connections 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea2c06b3d0 0x7fea2c1685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea18066f30 0x7fea180693e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea2c156e30 0x7fea2c0c5680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 --2- 172.21.15.16:0/3610117668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea2c1558f0 0x7fea2c147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 >> 172.21.15.16:0/3610117668 conn(0x7fea2c06c410 msgr2=0x7fea2c149130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 msgr2=0x7f88300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 0x7f88300693e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f884415c730 tx=0x7f8834009250 comp rx=0 tx=0).stop 2023-10-06T11:42:03.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 msgr2=0x7f8844159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 0x7f8844159dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f883800b530 tx=0x7f88380090f0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 shutdown_connections 2023-10-06T11:42:03.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7fea31652700 1 -- 172.21.15.16:0/3610117668 wait complete. 2023-10-06T11:42:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 shutdown_connections 2023-10-06T11:42:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f884406c5d0 0x7f8844159dd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8830066f30 0x7f88300693e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8844159240 0x7f884414cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 --2- 172.21.15.16:0/3670126329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88440ce1e0 0x7f88440b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 >> 172.21.15.16:0/3670126329 conn(0x7f884406d520 msgr2=0x7f884406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 shutdown_connections 2023-10-06T11:42:03.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:03.226+0000 7f8849752700 1 -- 172.21.15.16:0/3670126329 wait complete. 2023-10-06T11:42:03.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-06T11:42:03.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2023-10-06T11:42:04.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T11:42:04.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T11:42:04.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T11:42:04.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T11:42:04.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T11:42:04.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T11:42:04.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T11:42:04.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T11:42:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.390+0000 7f24ce702700 1 Processor -- start 2023-10-06T11:42:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.395+0000 7f24ce702700 1 -- start start 2023-10-06T11:42:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.396+0000 7f24ce702700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24c806e660 0x7f24c8099d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.396+0000 7f24ce702700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f24c8161890 0x7f24c8161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.396+0000 7f24ce702700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c8162230 con 0x7f24c8098780 2023-10-06T11:42:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.396+0000 7f24ce702700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c806f140 con 0x7f24c806e660 2023-10-06T11:42:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.396+0000 7f24ce702700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c815e200 con 0x7f24c8161890 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.397+0000 7f24c77fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f24c8098780 0x7f24c806ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48780/0 (socket says 172.21.15.16:48780) 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.397+0000 7f24c77fe700 1 -- 172.21.15.16:0/3957566825 learned_addr learned my addr 172.21.15.16:0/3957566825 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.397+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030467576 0 0) 0x7f24bc001110 con 0x7f24c8098780 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.397+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b0003570 con 0x7f24c8098780 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810029987 0 0) 0x7f24b0003570 con 0x7f24c8098780 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 >> v2:172.21.15.177:3300/0 conn(0x7f24c8161890 msgr2=0x7f24c8161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 --2- 172.21.15.16:0/3957566825 >> v2:172.21.15.177:3300/0 conn(0x7f24c8161890 0x7f24c8161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 >> v2:172.21.15.16:3300/0 conn(0x7f24c806e660 msgr2=0x7f24c8099d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> v2:172.21.15.16:3300/0 conn(0x7f24c806e660 0x7f24c8099d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 --2- 172.21.15.16:0/3957566825 >> v2:172.21.15.16:3300/0 conn(0x7f24c806e660 0x7f24c8099d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c8098c80 con 0x7f24c8098780 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f24bc001600 con 0x7f24c8098780 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 >> v1:172.21.15.177:6789/0 conn(0x7f24c8098780 legacy=0x7f24c806ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 0x7f24b0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0004810 0x7f24b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24b0009a50 0x7f24b000be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815e200 con 0x7f24b0003b60 2023-10-06T11:42:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b000c3f0 con 0x7f24b0009a50 2023-10-06T11:42:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b000c5b0 con 0x7f24b0004810 2023-10-06T11:42:04.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> v2:172.21.15.16:3300/0 conn(0x7f24c806e660 0x7f24c8099d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.398+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 0x7f24b0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 -- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0004810 msgr2=0x7f24b0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0004810 0x7f24b0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 -- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24b0009a50 msgr2=0x7f24b000be30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24b0009a50 0x7f24b000be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 -- 172.21.15.16:0/3957566825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0003570 con 0x7f24b0003b60 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c7fff700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 0x7f24b0003f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24b800abe0 tx=0x7f24b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea2367b7cce1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800c6d0 con 0x7f24b0003b60 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800c830 con 0x7f24b0003b60 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.399+0000 7f24c6ffd700 1 -- 172.21.15.16:0/3957566825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cbd0 con 0x7f24b0003b60 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.400+0000 7f24ccc9f700 1 -- 172.21.15.16:0/3957566825 reap_dead start 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.400+0000 7f24ce702700 1 -- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 msgr2=0x7f24b0003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.400+0000 7f24ce702700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 0x7f24b0003f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24b800abe0 tx=0x7f24b8008040 comp rx=0 tx=0).stop 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f24ce702700 1 -- 172.21.15.16:0/3957566825 shutdown_connections 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f24ce702700 1 --2- 172.21.15.16:0/3957566825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24b0003b60 0x7f24b0003f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f24ce702700 1 -- 172.21.15.16:0/3957566825 >> 172.21.15.16:0/3957566825 conn(0x7f24c80d1250 msgr2=0x7f24c80cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f523641b700 1 Processor -- start 2023-10-06T11:42:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f523641b700 1 -- start start 2023-10-06T11:42:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.401+0000 7f24ce702700 1 -- 172.21.15.16:0/3957566825 shutdown_connections 2023-10-06T11:42:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f523641b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5230159d00 0x7f52300c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f523641b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523006bc10 0x7f52300c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f523641b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523006ed50 con 0x7f5230159930 2023-10-06T11:42:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f523641b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300c7790 con 0x7f5230159d00 2023-10-06T11:42:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f523641b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523006e420 con 0x7f523006bc10 2023-10-06T11:42:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.402+0000 7f24ce702700 1 -- 172.21.15.16:0/3957566825 wait complete. 2023-10-06T11:42:04.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.403+0000 7f522f7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5230159d00 0x7f52300c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.403+0000 7f522f7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5230159d00 0x7f52300c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45254/0 (socket says 172.21.15.16:45254) 2023-10-06T11:42:04.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.403+0000 7f522f7fe700 1 -- 172.21.15.16:0/999273388 learned_addr learned my addr 172.21.15.16:0/999273388 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.403+0000 7f52349b8700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523006bc10 0x7f52300c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.403+0000 7f24ce702700 1 Processor -- start 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877191886 0 0) 0x7f523006ed50 con 0x7f5230159930 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5218003570 con 0x7f5230159930 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2887149028 0 0) 0x7f5218003570 con 0x7f5230159930 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523006bc10 msgr2=0x7f52300c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f523006bc10 0x7f52300c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 >> v2:172.21.15.177:3300/0 conn(0x7f5230159d00 msgr2=0x7f52300c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 --2- 172.21.15.16:0/999273388 >> v2:172.21.15.177:3300/0 conn(0x7f5230159d00 0x7f52300c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523006e700 con 0x7f5230159930 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f5224001470 con 0x7f5230159930 2023-10-06T11:42:04.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 >> v1:172.21.15.16:6789/0 conn(0x7f5230159930 legacy=0x7f523006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5218003ae0 0x7f5218003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5218004810 0x7f5218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52180051c0 0x7f521800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c7790 con 0x7f5218004810 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523006e420 con 0x7f5218003ae0 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521800c0d0 con 0x7f52180051c0 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.404+0000 7f24ce702700 1 -- start start 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0003b60 0x7f24c815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c80ca360 0x7f24c80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815dce0 con 0x7f24c80c6d90 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80c9930 con 0x7f24c80ca360 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24ce702700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80c7200 con 0x7f24b0003b60 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f522f7fe700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5218004810 0x7f5218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24c77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43282/0 (socket says 172.21.15.16:43282) 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.405+0000 7f24c77fe700 1 -- 172.21.15.16:0/1242147167 learned_addr learned my addr 172.21.15.16:0/1242147167 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c7fff700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0003b60 0x7f24c815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24ccc9f700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c80ca360 0x7f24c80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 -- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52180051c0 msgr2=0x7f521800ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52180051c0 0x7f521800ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 -- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5218003ae0 msgr2=0x7f5218003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5218003ae0 0x7f5218003ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 -- 172.21.15.16:0/999273388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523006e700 con 0x7f5218004810 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c77fe700 1 -- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0003b60 msgr2=0x7f24c815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c77fe700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0003b60 0x7f24c815d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c77fe700 1 -- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c80ca360 msgr2=0x7f24c80aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c77fe700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c80ca360 0x7f24c80aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f24c77fe700 1 -- 172.21.15.16:0/1242147167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c81597a0 con 0x7f24c80c6d90 2023-10-06T11:42:04.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522f7fe700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5218004810 0x7f5218004c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5228009da0 tx=0x7f52280091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71f642adf576c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.406+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228009790 con 0x7f5218004810 2023-10-06T11:42:04.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.407+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800f040 con 0x7f5218004810 2023-10-06T11:42:04.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.407+0000 7f522effd700 1 -- 172.21.15.16:0/999273388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228007d80 con 0x7f5218004810 2023-10-06T11:42:04.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.407+0000 7f24c77fe700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00cb40 tx=0x7f24bc001c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f523641b700 1 -- 172.21.15.16:0/999273388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5218004810 msgr2=0x7f5218004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f523641b700 1 --2- 172.21.15.16:0/999273388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5218004810 0x7f5218004c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5228009da0 tx=0x7f52280091c0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc007430 con 0x7f24c80c6d90 2023-10-06T11:42:04.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc007590 con 0x7f24c80c6d90 2023-10-06T11:42:04.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004a60 con 0x7f24c80c6d90 2023-10-06T11:42:04.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c815d720 con 0x7f24c80c6d90 2023-10-06T11:42:04.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.408+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c80c9b60 con 0x7f24c80c6d90 2023-10-06T11:42:04.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.410+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a80052b0 con 0x7f24c80c6d90 2023-10-06T11:42:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.415+0000 7f52349b8700 1 -- 172.21.15.16:0/999273388 reap_dead start 2023-10-06T11:42:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.415+0000 7f523641b700 1 -- 172.21.15.16:0/999273388 shutdown_connections 2023-10-06T11:42:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.415+0000 7f523641b700 1 -- 172.21.15.16:0/999273388 >> 172.21.15.16:0/999273388 conn(0x7f523006c410 msgr2=0x7f523006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.417+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24bc00a660 con 0x7f24c80c6d90 2023-10-06T11:42:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.418+0000 7f24affff700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 0x7f24b4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.418+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f24bc084340 con 0x7f24c80c6d90 2023-10-06T11:42:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.418+0000 7f523641b700 1 -- 172.21.15.16:0/999273388 shutdown_connections 2023-10-06T11:42:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.418+0000 7f24c7fff700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 0x7f24b4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.419+0000 7f24c7fff700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 0x7f24b4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24b800abe0 tx=0x7f24b8019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.424+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24bc0514b0 con 0x7f24c80c6d90 2023-10-06T11:42:04.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.425+0000 7f523641b700 1 -- 172.21.15.16:0/999273388 wait complete. 2023-10-06T11:42:04.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.426+0000 7f523641b700 1 Processor -- start 2023-10-06T11:42:04.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.427+0000 7f523641b700 1 -- start start 2023-10-06T11:42:04.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b18b0 0x7f52300ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52300b1c80 0x7f5230158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300cbee0 con 0x7f52300b1c80 2023-10-06T11:42:04.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300cc370 con 0x7f5230158650 2023-10-06T11:42:04.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f523641b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300cb5b0 con 0x7f52300b18b0 2023-10-06T11:42:04.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f52349b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f52349b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45266/0 (socket says 172.21.15.16:45266) 2023-10-06T11:42:04.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.428+0000 7f52349b8700 1 -- 172.21.15.16:0/1695334856 learned_addr learned my addr 172.21.15.16:0/1695334856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 -- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b18b0 msgr2=0x7f52300ccdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b18b0 0x7f52300ccdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 -- 172.21.15.16:0/1695334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52300b1c80 msgr2=0x7f5230158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52300b1c80 0x7f5230158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 -- 172.21.15.16:0/1695334856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300cb890 con 0x7f5230158650 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f52349b8700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f523006bfe0 tx=0x7f5220008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220017070 con 0x7f5230158650 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52200125f0 con 0x7f5230158650 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200129b0 con 0x7f5230158650 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52300cbb70 con 0x7f5230158650 2023-10-06T11:42:04.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.429+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52300cd2f0 con 0x7f5230158650 2023-10-06T11:42:04.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.430+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52300cbd00 con 0x7f5230158650 2023-10-06T11:42:04.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.431+0000 7f403906d700 1 Processor -- start 2023-10-06T11:42:04.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.431+0000 7f403906d700 1 -- start start 2023-10-06T11:42:04.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f403906d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4034067ee0 0x7f403406b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f403906d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403406af60 con 0x7f4034159970 2023-10-06T11:42:04.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f403906d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40340c90c0 con 0x7f4034069ec0 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f403906d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40340c7710 con 0x7f4034067ee0 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f4032d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4034159970 0x7f40340c89b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57032/0 (socket says 172.21.15.16:57032) 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.432+0000 7f4032d9d700 1 -- 172.21.15.16:0/2480350509 learned_addr learned my addr 172.21.15.16:0/2480350509 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.433+0000 7f403359e700 1 --2- 172.21.15.16:0/2480350509 >> v2:172.21.15.16:3300/0 conn(0x7f4034067ee0 0x7f403406b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.433+0000 7f403359e700 1 -- 172.21.15.16:0/2480350509 >> v1:172.21.15.177:6789/0 conn(0x7f4034069ec0 legacy=0x7f40340677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.433+0000 7f403359e700 1 -- 172.21.15.16:0/2480350509 >> v1:172.21.15.16:6789/0 conn(0x7f4034159970 legacy=0x7f40340c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.433+0000 7f403359e700 1 -- 172.21.15.16:0/2480350509 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403406a350 con 0x7f4034067ee0 2023-10-06T11:42:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.433+0000 7f403359e700 1 --2- 172.21.15.16:0/2480350509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034067ee0 0x7f403406b3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f40280098e0 tx=0x7f40280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9923e91fa86441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.434+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5220013480 con 0x7f5230158650 2023-10-06T11:42:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.434+0000 7f5217fff700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 0x7f521c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.434+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5220084230 con 0x7f5230158650 2023-10-06T11:42:04.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.435+0000 7f522ffff700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 0x7f521c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.435+0000 7f522ffff700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 0x7f521c0693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5224007db0 tx=0x7f52240072e0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.437+0000 7ff1cadbe700 1 Processor -- start 2023-10-06T11:42:04.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.437+0000 7ff1cadbe700 1 -- start start 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1cadbe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1c40cbe20 0x7ff1c4150d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1cadbe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40cc990 con 0x7ff1c415a2a0 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1cadbe700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c4157e00 con 0x7ff1c4150960 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1cadbe700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c415a780 con 0x7ff1c40cbe20 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1c3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1c415a2a0 0x7ff1c40cb710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57046/0 (socket says 172.21.15.16:57046) 2023-10-06T11:42:04.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.438+0000 7ff1c3fff700 1 -- 172.21.15.16:0/2457707259 learned_addr learned my addr 172.21.15.16:0/2457707259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310908048 0 0) 0x7ff1c40cc990 con 0x7ff1c415a2a0 2023-10-06T11:42:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b4003570 con 0x7ff1c415a2a0 2023-10-06T11:42:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427132563 0 0) 0x7ff1c4157e00 con 0x7ff1c4150960 2023-10-06T11:42:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c40cc990 con 0x7ff1c4150960 2023-10-06T11:42:04.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5220051340 con 0x7f5230158650 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744738931 0 0) 0x7ff1c40cc990 con 0x7ff1c4150960 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 >> v2:172.21.15.177:3300/0 conn(0x7ff1c40cbe20 msgr2=0x7ff1c4150d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 --2- 172.21.15.16:0/2457707259 >> v2:172.21.15.177:3300/0 conn(0x7ff1c40cbe20 0x7ff1c4150d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 >> v1:172.21.15.16:6789/0 conn(0x7ff1c415a2a0 legacy=0x7ff1c40cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7f4031d9b700 1 -- 172.21.15.16:0/2480350509 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028009e80 con 0x7f4034067ee0 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7f4031d9b700 1 -- 172.21.15.16:0/2480350509 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402800d970 con 0x7f4034067ee0 2023-10-06T11:42:04.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7f4031d9b700 1 -- 172.21.15.16:0/2480350509 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800dd10 con 0x7f4034067ee0 2023-10-06T11:42:04.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c415aa60 con 0x7ff1c4150960 2023-10-06T11:42:04.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7ff1b8001470 con 0x7ff1c4150960 2023-10-06T11:42:04.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 >> v1:172.21.15.177:6789/0 conn(0x7ff1c4150960 legacy=0x7ff1c41576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1b4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4004810 0x7ff1b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4005310 0x7ff1b400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c415a780 con 0x7ff1b4003bf0 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b400e440 con 0x7ff1b4005310 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b400e600 con 0x7ff1b4004810 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.439+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1b4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4004810 msgr2=0x7ff1b4004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1b4004810 0x7ff1b4004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4005310 msgr2=0x7ff1b400ba60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1b4005310 0x7ff1b400ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/2457707259 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c415aa60 con 0x7ff1b4003bf0 2023-10-06T11:42:04.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1b4003fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c2f0 tx=0x7ff1b8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5361d6fc2b4c74f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7f403906d700 1 -- 172.21.15.16:0/2480350509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034067ee0 msgr2=0x7f403406b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7f403906d700 1 --2- 172.21.15.16:0/2480350509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034067ee0 0x7f403406b3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f40280098e0 tx=0x7f40280091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800cb40 con 0x7ff1b4003bf0 2023-10-06T11:42:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b800cca0 con 0x7ff1b4003bf0 2023-10-06T11:42:04.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7ff1c37fe700 1 -- 172.21.15.16:0/2457707259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8012420 con 0x7ff1b4003bf0 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7f403906d700 1 -- 172.21.15.16:0/2480350509 shutdown_connections 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7f403906d700 1 --2- 172.21.15.16:0/2480350509 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034067ee0 0x7f403406b3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.440+0000 7f403906d700 1 -- 172.21.15.16:0/2480350509 >> 172.21.15.16:0/2480350509 conn(0x7f403406ce90 msgr2=0x7f40340cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.441+0000 7f403906d700 1 -- 172.21.15.16:0/2480350509 shutdown_connections 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.441+0000 7ff1c935b700 1 -- 172.21.15.16:0/2457707259 reap_dead start 2023-10-06T11:42:04.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.441+0000 7ff1cadbe700 1 -- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 msgr2=0x7ff1b4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.441+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1b4003fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c2f0 tx=0x7ff1b8007830 comp rx=0 tx=0).stop 2023-10-06T11:42:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.441+0000 7f403906d700 1 -- 172.21.15.16:0/2480350509 wait complete. 2023-10-06T11:42:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.443+0000 7ff1cadbe700 1 -- 172.21.15.16:0/2457707259 shutdown_connections 2023-10-06T11:42:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.443+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/2457707259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1b4003fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.443+0000 7ff1cadbe700 1 -- 172.21.15.16:0/2457707259 >> 172.21.15.16:0/2457707259 conn(0x7ff1c4068350 msgr2=0x7ff1c4068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.443+0000 7f403906d700 1 Processor -- start 2023-10-06T11:42:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.443+0000 7f403906d700 1 -- start start 2023-10-06T11:42:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f6d9d3f5700 1 Processor -- start 2023-10-06T11:42:04.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7ff1cadbe700 1 -- 172.21.15.16:0/2457707259 shutdown_connections 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4034067ee0 0x7f40341544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4034159970 0x7f4034144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40341448b0 con 0x7f4034069ec0 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40341657a0 con 0x7f4034159970 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f403906d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4034165a80 con 0x7f4034067ee0 2023-10-06T11:42:04.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43316/0 (socket says 172.21.15.16:43316) 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 -- 172.21.15.16:0/75283153 learned_addr learned my addr 172.21.15.16:0/75283153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 -- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4034067ee0 msgr2=0x7f40341544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4034067ee0 0x7f40341544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 -- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4034159970 msgr2=0x7f4034144370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4034159970 0x7f4034144370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 -- 172.21.15.16:0/75283153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028009420 con 0x7f4034069ec0 2023-10-06T11:42:04.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f403259c700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f402c00bb80 tx=0x7f402c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7f6d9d3f5700 1 -- start start 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981554d0 0x7f6d980ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 0x7f6d980ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d9806a380 0x7f6d9806a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9806edd0 con 0x7f6d9806a380 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9806bc80 con 0x7f6d981558a0 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.445+0000 7f6d9d3f5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980aea80 con 0x7f6d981554d0 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d9806a380 0x7f6d9806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d9806a380 0x7f6d9806a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43326/0 (socket says 172.21.15.16:43326) 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 -- 172.21.15.16:0/1880241168 learned_addr learned my addr 172.21.15.16:0/1880241168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d977fe700 1 --2- 172.21.15.16:0/1880241168 >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 0x7f6d980ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.444+0000 7ff1cadbe700 1 -- 172.21.15.16:0/2457707259 wait complete. 2023-10-06T11:42:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7ff1cadbe700 1 Processor -- start 2023-10-06T11:42:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c00c970 con 0x7f4034069ec0 2023-10-06T11:42:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00cad0 con 0x7f4034069ec0 2023-10-06T11:42:04.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c00ce90 con 0x7f4034069ec0 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 -- 172.21.15.16:0/1880241168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981554d0 msgr2=0x7f6d980ae000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 --2- 172.21.15.16:0/1880241168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981554d0 0x7f6d980ae000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 -- 172.21.15.16:0/1880241168 >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 msgr2=0x7f6d980ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 --2- 172.21.15.16:0/1880241168 >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 0x7f6d980ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d9cbf4700 1 -- 172.21.15.16:0/1880241168 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9806b070 con 0x7f6d9806a380 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f6d977fe700 1 --2- 172.21.15.16:0/1880241168 >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 0x7f6d980ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4034165d60 con 0x7f4034069ec0 2023-10-06T11:42:04.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4034165480 con 0x7f4034069ec0 2023-10-06T11:42:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.447+0000 7f6d9cbf4700 1 --2- 172.21.15.16:0/1880241168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d9806a800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d980cd2c0 tx=0x7f6d9000c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab85ed911d6dff9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7f6d96ffd700 1 -- 172.21.15.16:0/1880241168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000be30 con 0x7f6d9806a380 2023-10-06T11:42:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7f6d96ffd700 1 -- 172.21.15.16:0/1880241168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9000bf90 con 0x7f6d9806a380 2023-10-06T11:42:04.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7f6d96ffd700 1 -- 172.21.15.16:0/1880241168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000c4c0 con 0x7f6d9806a380 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.446+0000 7ff1cadbe700 1 -- start start 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c416d9b0 0x7ff1c40c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c414bd70 0x7ff1c414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c4010 con 0x7ff1b4003bf0 2023-10-06T11:42:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c47c0 con 0x7ff1c416d9b0 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1cadbe700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c6410 con 0x7ff1c414bd70 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43338/0 (socket says 172.21.15.16:43338) 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/3692152466 learned_addr learned my addr 172.21.15.16:0/3692152466 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c935b700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c414bd70 0x7ff1c414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c3fff700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c416d9b0 0x7ff1c40c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c414bd70 msgr2=0x7ff1c414aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c414bd70 0x7ff1c414aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c416d9b0 msgr2=0x7ff1c40c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c416d9b0 0x7ff1c40c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7ff1c8b5a700 1 -- 172.21.15.16:0/3692152466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c416df20 con 0x7ff1b4003bf0 2023-10-06T11:42:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.448+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f402c014620 con 0x7f4034069ec0 2023-10-06T11:42:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7ff1c8b5a700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c8e0 tx=0x7ff1b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7f401affd700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 0x7f40200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f402c084790 con 0x7f4034069ec0 2023-10-06T11:42:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7f4032d9d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 0x7f40200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/1880241168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 msgr2=0x7f6d9806a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.449+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/1880241168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d9806a800 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d980cd2c0 tx=0x7f6d9000c3c0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800cee0 con 0x7ff1b4003bf0 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b80127d0 con 0x7ff1b4003bf0 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8012b50 con 0x7ff1b4003bf0 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c416e200 con 0x7ff1b4003bf0 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c416e7c0 con 0x7ff1b4003bf0 2023-10-06T11:42:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.450+0000 7f4032d9d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 0x7f40200693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f40240058d0 tx=0x7f4024007ac0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.451+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c4063e70 con 0x7ff1b4003bf0 2023-10-06T11:42:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.452+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1b8024070 con 0x7ff1b4003bf0 2023-10-06T11:42:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.452+0000 7ff1c0ff9700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 0x7ff1a80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.452+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1b8086650 con 0x7ff1b4003bf0 2023-10-06T11:42:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.453+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4034166260 con 0x7f4034069ec0 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.453+0000 7ff1c3fff700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 0x7ff1a80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.453+0000 7ff1c3fff700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 0x7ff1a80694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0007fd0 tx=0x7ff1b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.459+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/1880241168 shutdown_connections 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.459+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/1880241168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d9806a800 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.459+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/1880241168 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981554d0 0x7f6d980ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.459+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/1880241168 >> v2:172.21.15.177:3300/0 conn(0x7f6d981558a0 0x7f6d980ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.459+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/1880241168 >> 172.21.15.16:0/1880241168 conn(0x7f6d9806c410 msgr2=0x7f6d9806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.461+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1b80537e0 con 0x7ff1b4003bf0 2023-10-06T11:42:04.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.467+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/1880241168 shutdown_connections 2023-10-06T11:42:04.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.467+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/1880241168 wait complete. 2023-10-06T11:42:04.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.468+0000 7f6d9d3f5700 1 Processor -- start 2023-10-06T11:42:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.468+0000 7f6d9d3f5700 1 -- start start 2023-10-06T11:42:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 0x7f6d9814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 0x7f6d9814ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980affa0 con 0x7f6d9806a380 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980af0b0 con 0x7f6d981554d0 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9d3f5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980af390 con 0x7f6d981558a0 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d97fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43354/0 (socket says 172.21.15.16:43354) 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d97fff700 1 -- 172.21.15.16:0/3763425999 learned_addr learned my addr 172.21.15.16:0/3763425999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d9cbf4700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 0x7f6d9814ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f6d977fe700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 0x7f6d9814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.469+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f402c04c5a0 con 0x7f4034069ec0 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 -- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 msgr2=0x7f6d9814ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 0x7f6d9814ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 -- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 msgr2=0x7f6d9814a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 0x7f6d9814a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 -- 172.21.15.16:0/3763425999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90007460 con 0x7f6d9806a380 2023-10-06T11:42:04.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d977fe700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 0x7f6d9814a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d9cbf4700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 0x7f6d9814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.470+0000 7f6d97fff700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b5e0 tx=0x7f6d880090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.471+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800f510 con 0x7f6d9806a380 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.471+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800f690 con 0x7f6d9806a380 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.471+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800fa50 con 0x7f6d9806a380 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.471+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980af670 con 0x7f6d9806a380 2023-10-06T11:42:04.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.471+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d980afc30 con 0x7f6d9806a380 2023-10-06T11:42:04.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.473+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d8801a400 con 0x7f6d9806a380 2023-10-06T11:42:04.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.474+0000 7f6d87fff700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 0x7f6d7c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.474+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d88084310 con 0x7f6d9806a380 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.473+0000 7f82a5ce3700 1 Processor -- start 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.473+0000 7f82a5ce3700 1 -- start start 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f82a5ce3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f82a006cf20 0x7f82a00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f82a5ce3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 0x7f82a00c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f82a5ce3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00c95f0 con 0x7f82a01558f0 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f82a5ce3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a006af40 con 0x7f82a006cf20 2023-10-06T11:42:04.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f82a5ce3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0157000 con 0x7f82a006b3d0 2023-10-06T11:42:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d740052b0 con 0x7f6d9806a380 2023-10-06T11:42:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f829ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 0x7f82a00c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f829ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 0x7f82a00c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:41818/0 (socket says 172.21.15.16:41818) 2023-10-06T11:42:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f829ffff700 1 -- 172.21.15.16:0/3823117095 learned_addr learned my addr 172.21.15.16:0/3823117095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f6d977fe700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 0x7f6d7c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.475+0000 7f829effd700 1 --2- 172.21.15.16:0/3823117095 >> v2:172.21.15.177:3300/0 conn(0x7f82a006cf20 0x7f82a00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f829effd700 1 -- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 msgr2=0x7f82a00c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f829effd700 1 --2- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 0x7f82a00c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f829effd700 1 -- 172.21.15.16:0/3823117095 >> v1:172.21.15.177:6789/0 conn(0x7f82a01558f0 legacy=0x7f82a01568f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f829effd700 1 -- 172.21.15.16:0/3823117095 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a006a610 con 0x7f82a006cf20 2023-10-06T11:42:04.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f829effd700 1 --2- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006cf20 0x7f82a00c8b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f829000bb90 tx=0x7f829000b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71862a04f84a7123 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.476+0000 7f6d977fe700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 0x7f6d7c069500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c006f00 tx=0x7f6d8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.477+0000 7f829e7fc700 1 -- 172.21.15.16:0/3823117095 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000db00 con 0x7f82a006cf20 2023-10-06T11:42:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.477+0000 7f829e7fc700 1 -- 172.21.15.16:0/3823117095 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829000dc60 con 0x7f82a006cf20 2023-10-06T11:42:04.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.477+0000 7f829e7fc700 1 -- 172.21.15.16:0/3823117095 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290010470 con 0x7f82a006cf20 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 -- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006cf20 msgr2=0x7f82a00c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006cf20 0x7f82a00c8b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f829000bb90 tx=0x7f829000b430 comp rx=0 tx=0).stop 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 -- 172.21.15.16:0/3823117095 shutdown_connections 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006b3d0 0x7f82a00c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/3823117095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006cf20 0x7f82a00c8b70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 -- 172.21.15.16:0/3823117095 >> 172.21.15.16:0/3823117095 conn(0x7f82a006c410 msgr2=0x7f82a006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 -- 172.21.15.16:0/3823117095 shutdown_connections 2023-10-06T11:42:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.478+0000 7f82a5ce3700 1 -- 172.21.15.16:0/3823117095 wait complete. 2023-10-06T11:42:04.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.480+0000 7f82a5ce3700 1 Processor -- start 2023-10-06T11:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.480+0000 7f82a5ce3700 1 -- start start 2023-10-06T11:42:04.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.481+0000 7f82a5ce3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006b3d0 0x7f82a016f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.481+0000 7f82a5ce3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 0x7f82a00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f82a5ce3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f82a5ce3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a016f6c0 con 0x7f82a00cb220 2023-10-06T11:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f82a5ce3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00cced0 con 0x7f82a006b3d0 2023-10-06T11:42:04.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f82a5ce3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00ca460 con 0x7f82a006cf20 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f829ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f829ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43368/0 (socket says 172.21.15.16:43368) 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f829f7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006b3d0 0x7f82a016f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f829ffff700 1 -- 172.21.15.16:0/96971581 learned_addr learned my addr 172.21.15.16:0/96971581 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.482+0000 7f829effd700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 0x7f82a00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 -- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 msgr2=0x7f82a00cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 0x7f82a00cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 -- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006b3d0 msgr2=0x7f82a016f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006b3d0 0x7f82a016f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 -- 172.21.15.16:0/96971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829000b6d0 con 0x7f82a00cb220 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829effd700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 0x7f82a00cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.483+0000 7f829ffff700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8294002f40 tx=0x7f8294009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d88051430 con 0x7f6d9806a380 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294013680 con 0x7f82a00cb220 2023-10-06T11:42:04.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82940137e0 con 0x7f82a00cb220 2023-10-06T11:42:04.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294013b80 con 0x7f82a00cb220 2023-10-06T11:42:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00ca740 con 0x7f82a00cb220 2023-10-06T11:42:04.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.484+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a00cad00 con 0x7f82a00cb220 2023-10-06T11:42:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.489+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f829400c400 con 0x7f82a00cb220 2023-10-06T11:42:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.490+0000 7f82877fe700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 0x7f828c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.490+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82940847c0 con 0x7f82a00cb220 2023-10-06T11:42:04.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.491+0000 7f829f7fe700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 0x7f828c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.491+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82800052b0 con 0x7f82a00cb220 2023-10-06T11:42:04.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.493+0000 7f829f7fe700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 0x7f828c069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82a0069960 tx=0x7f8298009780 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.500+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82940518b0 con 0x7f82a00cb220 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.528+0000 7efcc61e5700 1 Processor -- start 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.528+0000 7efcc61e5700 1 -- start start 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcc61e5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efcc0169bd0 0x7efcc00c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcc61e5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc00c94d0 0x7efcc006a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcc61e5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00c6b30 con 0x7efcc00c7720 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcc61e5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc00c6e10 con 0x7efcc0169bd0 2023-10-06T11:42:04.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcc61e5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0069fc0 con 0x7efcc00c94d0 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.529+0000 7efcbeffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efcc00c7720 0x7efcc00c8dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57048/0 (socket says 172.21.15.16:57048) 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.530+0000 7efcbeffd700 1 -- 172.21.15.16:0/1376900250 learned_addr learned my addr 172.21.15.16:0/1376900250 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.530+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319977600 0 0) 0x7efcc00c6b30 con 0x7efcc00c7720 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.530+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca8003570 con 0x7efcc00c7720 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108371840 0 0) 0x7efca8003570 con 0x7efcc00c7720 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc00c94d0 msgr2=0x7efcc006a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc00c94d0 0x7efcc006a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 >> v2:172.21.15.177:3300/0 conn(0x7efcc0169bd0 msgr2=0x7efcc00c1c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 --2- 172.21.15.16:0/1376900250 >> v2:172.21.15.177:3300/0 conn(0x7efcc0169bd0 0x7efcc00c1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc006a2a0 con 0x7efcc00c7720 2023-10-06T11:42:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7efcb80017f0 con 0x7efcc00c7720 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 >> v1:172.21.15.16:6789/0 conn(0x7efcc00c7720 legacy=0x7efcc00c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efca8004810 0x7efca8004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca80037b0 0x7efca8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efca8004360 0x7efca8005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c6e10 con 0x7efca80037b0 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0069fc0 con 0x7efca8004360 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca80062b0 con 0x7efca8004810 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.531+0000 7efcbeffd700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca80037b0 0x7efca8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 -- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efca8004810 msgr2=0x7efca8004f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efca8004810 0x7efca8004f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 -- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efca8004360 msgr2=0x7efca8005a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efca8004360 0x7efca8005a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 -- 172.21.15.16:0/1376900250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8003570 con 0x7efca80037b0 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbeffd700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca80037b0 0x7efca8003c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcc0069c20 tx=0x7efcc006cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab990cdeec01245 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011760 con 0x7efca80037b0 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb80118c0 con 0x7efca80037b0 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.532+0000 7efcbe7fc700 1 -- 172.21.15.16:0/1376900250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8011c60 con 0x7efca80037b0 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.533+0000 7efcc61e5700 1 -- 172.21.15.16:0/1376900250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca80037b0 msgr2=0x7efca8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.533+0000 7efcc61e5700 1 --2- 172.21.15.16:0/1376900250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca80037b0 0x7efca8003c20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcc0069c20 tx=0x7efcc006cf20 comp rx=0 tx=0).stop 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.534+0000 7efcbffff700 1 -- 172.21.15.16:0/1376900250 reap_dead start 2023-10-06T11:42:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.534+0000 7efcc61e5700 1 -- 172.21.15.16:0/1376900250 shutdown_connections 2023-10-06T11:42:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.534+0000 7efcc61e5700 1 -- 172.21.15.16:0/1376900250 >> 172.21.15.16:0/1376900250 conn(0x7efcc006c410 msgr2=0x7efcc006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.534+0000 7efcc61e5700 1 -- 172.21.15.16:0/1376900250 shutdown_connections 2023-10-06T11:42:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.534+0000 7efcc61e5700 1 -- 172.21.15.16:0/1376900250 wait complete. 2023-10-06T11:42:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.536+0000 7efcc61e5700 1 Processor -- start 2023-10-06T11:42:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.536+0000 7efcc61e5700 1 -- start start 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcc016c730 0x7efcc00c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc0161c00 0x7efcc00b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c59b0 con 0x7efcc016cb00 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c6440 con 0x7efcc016c730 2023-10-06T11:42:04.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcc61e5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c5ba0 con 0x7efcc0161c00 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcbeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcbeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43388/0 (socket says 172.21.15.16:43388) 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.537+0000 7efcbeffd700 1 -- 172.21.15.16:0/3168866779 learned_addr learned my addr 172.21.15.16:0/3168866779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 -- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc0161c00 msgr2=0x7efcc00b1aa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc0161c00 0x7efcc00b1aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 -- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcc016c730 msgr2=0x7efcc00c3470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcc016c730 0x7efcc00c3470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 -- 172.21.15.16:0/3168866779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc00c5e80 con 0x7efcc016cb00 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.538+0000 7efcbeffd700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efca8006200 tx=0x7efcb8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.539+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80016f0 con 0x7efcc016cb00 2023-10-06T11:42:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.539+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb8020040 con 0x7efcc016cb00 2023-10-06T11:42:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.539+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80070d0 con 0x7efcc016cb00 2023-10-06T11:42:04.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.540+0000 7faea3f46700 1 Processor -- start 2023-10-06T11:42:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.539+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc00c6160 con 0x7efcc016cb00 2023-10-06T11:42:04.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.539+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc00c39b0 con 0x7efcc016cb00 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.542+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcb8017070 con 0x7efcc016cb00 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.543+0000 7efca77fe700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 0x7efcac0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.543+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efcb808a1a0 con 0x7efcc016cb00 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.540+0000 7faea3f46700 1 -- start start 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.541+0000 7faea3f46700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae9c06b3d0 0x7fae9c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.541+0000 7faea3f46700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c06af40 con 0x7fae9c1558f0 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.541+0000 7faea3f46700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c157000 con 0x7fae9c155cc0 2023-10-06T11:42:04.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.541+0000 7faea3f46700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9c06a610 con 0x7fae9c06b3d0 2023-10-06T11:42:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.544+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc016a900 con 0x7efcc016cb00 2023-10-06T11:42:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.544+0000 7faea24e3700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fae9c155cc0 0x7fae9c0c6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:48828/0 (socket says 172.21.15.16:48828) 2023-10-06T11:42:04.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.544+0000 7faea24e3700 1 -- 172.21.15.16:0/2736905857 learned_addr learned my addr 172.21.15.16:0/2736905857 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:04.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.551+0000 7faea14e1700 1 --2- 172.21.15.16:0/2736905857 >> v2:172.21.15.16:3300/0 conn(0x7fae9c06b3d0 0x7fae9c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.551+0000 7efcbf7fe700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 0x7efcac0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.551+0000 7efcbf7fe700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 0x7efcac0694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efcc006d780 tx=0x7efcb0006c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.553+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcb8057310 con 0x7efcc016cb00 2023-10-06T11:42:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.557+0000 7faea14e1700 1 -- 172.21.15.16:0/2736905857 >> v1:172.21.15.177:6789/0 conn(0x7fae9c155cc0 legacy=0x7fae9c0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.557+0000 7faea14e1700 1 -- 172.21.15.16:0/2736905857 >> v1:172.21.15.16:6789/0 conn(0x7fae9c1558f0 legacy=0x7fae9c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:04.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.557+0000 7faea14e1700 1 -- 172.21.15.16:0/2736905857 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c06a8f0 con 0x7fae9c06b3d0 2023-10-06T11:42:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.572+0000 7faea14e1700 1 --2- 172.21.15.16:0/2736905857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c06b3d0 0x7fae9c0c8b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae94009b50 tx=0x7fae94009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09a6f8d53bd0dde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.579+0000 7faea0ce0700 1 -- 172.21.15.16:0/2736905857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400d9b0 con 0x7fae9c06b3d0 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.579+0000 7faea0ce0700 1 -- 172.21.15.16:0/2736905857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400db10 con 0x7fae9c06b3d0 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.579+0000 7faea0ce0700 1 -- 172.21.15.16:0/2736905857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400deb0 con 0x7fae9c06b3d0 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.580+0000 7faea3f46700 1 -- 172.21.15.16:0/2736905857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c06b3d0 msgr2=0x7fae9c0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.580+0000 7faea3f46700 1 --2- 172.21.15.16:0/2736905857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c06b3d0 0x7fae9c0c8b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae94009b50 tx=0x7fae94009c30 comp rx=0 tx=0).stop 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.583+0000 7faea3f46700 1 -- 172.21.15.16:0/2736905857 shutdown_connections 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.583+0000 7faea3f46700 1 --2- 172.21.15.16:0/2736905857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c06b3d0 0x7fae9c0c8b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.583+0000 7faea3f46700 1 -- 172.21.15.16:0/2736905857 >> 172.21.15.16:0/2736905857 conn(0x7fae9c06c410 msgr2=0x7fae9c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.584+0000 7faea3f46700 1 -- 172.21.15.16:0/2736905857 shutdown_connections 2023-10-06T11:42:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.584+0000 7faea3f46700 1 -- 172.21.15.16:0/2736905857 wait complete. 2023-10-06T11:42:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.585+0000 7faea3f46700 1 Processor -- start 2023-10-06T11:42:04.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.586+0000 7faea3f46700 1 -- start start 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c06b3d0 0x7fae9c1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c155cc0 0x7fae9c1878b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1548f0 con 0x7fae9c1558f0 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c157820 con 0x7fae9c155cc0 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea3f46700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c153f20 con 0x7fae9c06b3d0 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea14e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea14e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43400/0 (socket says 172.21.15.16:43400) 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.587+0000 7faea14e1700 1 -- 172.21.15.16:0/1731458906 learned_addr learned my addr 172.21.15.16:0/1731458906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 -- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c06b3d0 msgr2=0x7fae9c1543b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:04.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c06b3d0 0x7fae9c1543b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 -- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c155cc0 msgr2=0x7fae9c1878b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c155cc0 0x7fae9c1878b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 -- 172.21.15.16:0/1731458906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94009790 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.588+0000 7faea14e1700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae9c06d460 tx=0x7fae94018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.589+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9401f070 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.589+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae94023400 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.589+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94010ea0 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.589+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c153930 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.589+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c0c3d20 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.593+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae940239d0 con 0x7fae9c1558f0 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.593+0000 7fae91ffb700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 0x7fae80069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:04.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.593+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fae94086160 con 0x7fae9c1558f0 2023-10-06T11:42:04.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.595+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c0052b0 con 0x7fae9c1558f0 2023-10-06T11:42:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.595+0000 7faea1ce2700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 0x7fae80069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:04.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.604+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae94053250 con 0x7fae9c1558f0 2023-10-06T11:42:04.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.623+0000 7faea1ce2700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 0x7fae80069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae98009740 tx=0x7fae98006c60 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.792+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f24a8005830 con 0x7f24c80c6d90 2023-10-06T11:42:04.799 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.794+0000 7f24affff700 1 -- 172.21.15.16:0/1242147167 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24bc018020 con 0x7f24c80c6d90 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 msgr2=0x7f24b4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 0x7f24b4069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24b800abe0 tx=0x7f24b8019000 comp rx=0 tx=0).stop 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 msgr2=0x7f24c8159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00cb40 tx=0x7f24bc001c90 comp rx=0 tx=0).stop 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 shutdown_connections 2023-10-06T11:42:04.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f24b4067080 0x7f24b4069530 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24b800abe0 tx=0x7f24b8019000 comp rx=0 tx=0).stop 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24c80ca360 0x7f24c80aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24b0003b60 0x7f24c815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 --2- 172.21.15.16:0/1242147167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c80c6d90 0x7f24c8159260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.797+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 >> 172.21.15.16:0/1242147167 conn(0x7f24c80d1250 msgr2=0x7f24c80ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.798+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 shutdown_connections 2023-10-06T11:42:04.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.798+0000 7f24ce702700 1 -- 172.21.15.16:0/1242147167 wait complete. 2023-10-06T11:42:04.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2023-10-06T11:42:04.818 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.848+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff1c415ad40 con 0x7ff1b4003bf0 2023-10-06T11:42:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.848+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5230063e70 con 0x7f5230158650 2023-10-06T11:42:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.848+0000 7ff1c0ff9700 1 -- 172.21.15.16:0/3692152466 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1b8056e00 con 0x7ff1b4003bf0 2023-10-06T11:42:04.851 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.849+0000 7f5217fff700 1 -- 172.21.15.16:0/1695334856 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5220054960 con 0x7f5230158650 2023-10-06T11:42:04.851 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-10-06T11:42:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 msgr2=0x7ff1a80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 0x7ff1a80694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0007fd0 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2023-10-06T11:42:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 msgr2=0x7ff1c40c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800c8e0 tx=0x7ff1b8017000 comp rx=0 tx=0).stop 2023-10-06T11:42:04.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 shutdown_connections 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1b4003bf0 0x7ff1c40c4280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1a8067000 0x7ff1a80694b0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0007fd0 tx=0x7ff1b0009040 comp rx=0 tx=0).stop 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1c414bd70 0x7ff1c414aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 --2- 172.21.15.16:0/3692152466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1c416d9b0 0x7ff1c40c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.852+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 >> 172.21.15.16:0/3692152466 conn(0x7ff1c4068350 msgr2=0x7ff1c406a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.853+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 shutdown_connections 2023-10-06T11:42:04.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.856+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 msgr2=0x7f521c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.856+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 0x7f521c0693e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5224007db0 tx=0x7f52240072e0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.856+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 msgr2=0x7f52300ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.856+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f523006bfe0 tx=0x7f5220008040 comp rx=0 tx=0).stop 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 shutdown_connections 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f521c066f30 0x7f521c0693e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5230158650 0x7f52300ca2c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f52300b18b0 0x7f52300ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 --2- 172.21.15.16:0/1695334856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52300b1c80 0x7f5230158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 >> 172.21.15.16:0/1695334856 conn(0x7f523006c410 msgr2=0x7f52300af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 shutdown_connections 2023-10-06T11:42:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.857+0000 7f523641b700 1 -- 172.21.15.16:0/1695334856 wait complete. 2023-10-06T11:42:04.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.858+0000 7ff1cadbe700 1 -- 172.21.15.16:0/3692152466 wait complete. 2023-10-06T11:42:04.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.875+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4034063e70 con 0x7f4034069ec0 2023-10-06T11:42:04.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T11:42:04.879 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-06T11:42:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.876+0000 7f401affd700 1 -- 172.21.15.16:0/75283153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f402c055330 con 0x7f4034069ec0 2023-10-06T11:42:04.880 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:04.880 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 msgr2=0x7f40200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 0x7f40200693e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f40240058d0 tx=0x7f4024007ac0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 msgr2=0x7f4034143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f402c00bb80 tx=0x7f402c009810 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 shutdown_connections 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4020066f30 0x7f40200693e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4034159970 0x7f4034144370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4034067ee0 0x7f40341544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.879+0000 7f403906d700 1 --2- 172.21.15.16:0/75283153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4034069ec0 0x7f4034143e30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.880+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 >> 172.21.15.16:0/75283153 conn(0x7f403406ce90 msgr2=0x7f40340cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.880+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 shutdown_connections 2023-10-06T11:42:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.880+0000 7f403906d700 1 -- 172.21.15.16:0/75283153 wait complete. 2023-10-06T11:42:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.883+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8280005560 con 0x7f82a00cb220 2023-10-06T11:42:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.884+0000 7f82877fe700 1 -- 172.21.15.16:0/96971581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8294021020 con 0x7f82a00cb220 2023-10-06T11:42:04.886 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.887+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 msgr2=0x7f828c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.887+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 0x7f828c069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82a0069960 tx=0x7f8298009780 comp rx=0 tx=0).stop 2023-10-06T11:42:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.887+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 msgr2=0x7f82a0144f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.887+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8294002f40 tx=0x7f8294009250 comp rx=0 tx=0).stop 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 shutdown_connections 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a00cb220 0x7f82a0144f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f828c067080 0x7f828c069530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82a006b3d0 0x7f82a016f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 --2- 172.21.15.16:0/96971581 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82a006cf20 0x7f82a00cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 >> 172.21.15.16:0/96971581 conn(0x7f82a006c410 msgr2=0x7f82a00c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 shutdown_connections 2023-10-06T11:42:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.888+0000 7f82a5ce3700 1 -- 172.21.15.16:0/96971581 wait complete. 2023-10-06T11:42:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.891+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6d74005560 con 0x7f6d9806a380 2023-10-06T11:42:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.891+0000 7f6d87fff700 1 -- 172.21.15.16:0/3763425999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d88054a50 con 0x7f6d9806a380 2023-10-06T11:42:04.893 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:04.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 msgr2=0x7f6d7c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 0x7f6d7c069500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c006f00 tx=0x7f6d8c008040 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 msgr2=0x7f6d981579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800b5e0 tx=0x7f6d880090f0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 shutdown_connections 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9806a380 0x7f6d981579e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6d7c067050 0x7f6d7c069500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d981558a0 0x7f6d9814ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 --2- 172.21.15.16:0/3763425999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d981554d0 0x7f6d9814a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.894+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 >> 172.21.15.16:0/3763425999 conn(0x7f6d9806c410 msgr2=0x7f6d98159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.895+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 shutdown_connections 2023-10-06T11:42:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.895+0000 7f6d9d3f5700 1 -- 172.21.15.16:0/3763425999 wait complete. 2023-10-06T11:42:04.899 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-06T11:42:04.900 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-06T11:42:04.905 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-06T11:42:04.910 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.914+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efcc00c15f0 con 0x7efcc016cb00 2023-10-06T11:42:04.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.914+0000 7efca77fe700 1 -- 172.21.15.16:0/3168866779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efcb805a930 con 0x7efcc016cb00 2023-10-06T11:42:04.916 INFO:teuthology.orchestra.run.smithi016.stdout:55834574855 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 msgr2=0x7efcac0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 0x7efcac0694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efcc006d780 tx=0x7efcb0006c60 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 msgr2=0x7efcc00b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efca8006200 tx=0x7efcb8017000 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 shutdown_connections 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efcac067000 0x7efcac0694b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efcc0161c00 0x7efcc00b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efcc016c730 0x7efcc00c3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 --2- 172.21.15.16:0/3168866779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcc016cb00 0x7efcc00b1560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 >> 172.21.15.16:0/3168866779 conn(0x7efcc006c410 msgr2=0x7efcc006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 shutdown_connections 2023-10-06T11:42:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.916+0000 7efcc61e5700 1 -- 172.21.15.16:0/3168866779 wait complete. 2023-10-06T11:42:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.924+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae7c005560 con 0x7fae9c1558f0 2023-10-06T11:42:04.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.925+0000 7fae91ffb700 1 -- 172.21.15.16:0/1731458906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae94056870 con 0x7fae9c1558f0 2023-10-06T11:42:04.927 INFO:teuthology.orchestra.run.smithi016.stdout:55834574854 2023-10-06T11:42:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 msgr2=0x7fae80069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 0x7fae80069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fae98009740 tx=0x7fae98006c60 comp rx=0 tx=0).stop 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 msgr2=0x7fae9c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fae9c06d460 tx=0x7fae94018000 comp rx=0 tx=0).stop 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 shutdown_connections 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fae80067080 0x7fae80069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae9c155cc0 0x7fae9c1878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fae9c06b3d0 0x7fae9c1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 --2- 172.21.15.16:0/1731458906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae9c1558f0 0x7fae9c1572e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 >> 172.21.15.16:0/1731458906 conn(0x7fae9c06c410 msgr2=0x7fae9c0cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 shutdown_connections 2023-10-06T11:42:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:04.927+0000 7faea3f46700 1 -- 172.21.15.16:0/1731458906 wait complete. 2023-10-06T11:42:04.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-10-06T11:42:04.932 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-06T11:42:04.941 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:04.941 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T11:42:04.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.100+0000 7f61a6505700 1 Processor -- start 2023-10-06T11:42:05.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.100+0000 7f61a6505700 1 -- start start 2023-10-06T11:42:05.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f61a6505700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61a01595b0 0x7f61a00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f61a6505700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f61a0159980 0x7f61a00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f61a6505700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00cf780 con 0x7f61a00cec70 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f61a6505700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00cbf20 con 0x7f61a01595b0 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f61a6505700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00cc200 con 0x7f61a0159980 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61a01595b0 0x7f61a00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61a01595b0 0x7f61a00cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43408/0 (socket says 172.21.15.16:43408) 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 -- 172.21.15.16:0/4135943484 learned_addr learned my addr 172.21.15.16:0/4135943484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:05.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619f7fe700 1 --2- 172.21.15.16:0/4135943484 >> v2:172.21.15.177:3300/0 conn(0x7f61a0159980 0x7f61a00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 -- 172.21.15.16:0/4135943484 >> v2:172.21.15.177:3300/0 conn(0x7f61a0159980 msgr2=0x7f61a00cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 --2- 172.21.15.16:0/4135943484 >> v2:172.21.15.177:3300/0 conn(0x7f61a0159980 0x7f61a00cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 -- 172.21.15.16:0/4135943484 >> v1:172.21.15.177:6789/0 conn(0x7f61a00cec70 legacy=0x7f61a00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 -- 172.21.15.16:0/4135943484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00cc4e0 con 0x7f61a01595b0 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.101+0000 7f619ffff700 1 --2- 172.21.15.16:0/4135943484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a01595b0 0x7f61a00cd140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61a0089c70 tx=0x7f6190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0a2cf585f2fcda server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f619effd700 1 -- 172.21.15.16:0/4135943484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000b820 con 0x7f61a01595b0 2023-10-06T11:42:05.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f619effd700 1 -- 172.21.15.16:0/4135943484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619000b980 con 0x7f61a01595b0 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f619effd700 1 -- 172.21.15.16:0/4135943484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619000bd20 con 0x7f61a01595b0 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 -- 172.21.15.16:0/4135943484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a01595b0 msgr2=0x7f61a00cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 --2- 172.21.15.16:0/4135943484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a01595b0 0x7f61a00cd140 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f61a0089c70 tx=0x7f6190009250 comp rx=0 tx=0).stop 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 -- 172.21.15.16:0/4135943484 shutdown_connections 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 --2- 172.21.15.16:0/4135943484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a01595b0 0x7f61a00cd140 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 --2- 172.21.15.16:0/4135943484 >> v2:172.21.15.177:3300/0 conn(0x7f61a0159980 0x7f61a00cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 -- 172.21.15.16:0/4135943484 >> 172.21.15.16:0/4135943484 conn(0x7f61a0099b40 msgr2=0x7f61a0087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 -- 172.21.15.16:0/4135943484 shutdown_connections 2023-10-06T11:42:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.102+0000 7f61a6505700 1 -- 172.21.15.16:0/4135943484 wait complete. 2023-10-06T11:42:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.103+0000 7f61a6505700 1 Processor -- start 2023-10-06T11:42:05.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.103+0000 7f61a6505700 1 -- start start 2023-10-06T11:42:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61a01595b0 0x7f61a00b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a0159980 0x7f61a0152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a0154450 con 0x7f61a00cec70 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00b4230 con 0x7f61a0159980 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a6505700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a00b3660 con 0x7f61a01595b0 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43414/0 (socket says 172.21.15.16:43414) 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 -- 172.21.15.16:0/541627660 learned_addr learned my addr 172.21.15.16:0/541627660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f61a4aa2700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a0159980 0x7f61a0152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619f7fe700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61a01595b0 0x7f61a00b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 -- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61a01595b0 msgr2=0x7f61a00b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61a01595b0 0x7f61a00b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 -- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a0159980 msgr2=0x7f61a0152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a0159980 0x7f61a0152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.104+0000 7f619ffff700 1 -- 172.21.15.16:0/541627660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61900094f0 con 0x7f61a00cec70 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f619ffff700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61900030c0 tx=0x7f6190016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190011e60 con 0x7f61a00cec70 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619001b680 con 0x7f61a00cec70 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a00c8c80 con 0x7f61a00cec70 2023-10-06T11:42:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619001ba20 con 0x7f61a00cec70 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.105+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a00b3be0 con 0x7f61a00cec70 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.106+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a00b4510 con 0x7f61a00cec70 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.106+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f619001c450 con 0x7f61a00cec70 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.106+0000 7f6187fff700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 0x7f618c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.107+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f61900863b0 con 0x7f61a00cec70 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.107+0000 7f619f7fe700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 0x7f618c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.107+0000 7f619f7fe700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 0x7f618c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61a016d0b0 tx=0x7f6194008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.111+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61900534c0 con 0x7f61a00cec70 2023-10-06T11:42:05.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.304+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61a00b4370 con 0x7f618c066f30 2023-10-06T11:42:05.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.313+0000 7f6187fff700 1 -- 172.21.15.16:0/541627660 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286837 (secure 0 0 0) 0x7f6194009640 con 0x7f618c066f30 2023-10-06T11:42:05.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:05.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 msgr2=0x7f618c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 0x7f618c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f61a016d0b0 tx=0x7f6194008040 comp rx=0 tx=0).stop 2023-10-06T11:42:05.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 msgr2=0x7f61a00c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f61900030c0 tx=0x7f6190016000 comp rx=0 tx=0).stop 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 shutdown_connections 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61a00cec70 0x7f61a00c8740 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f618c066f30 0x7f618c0693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f61a0159980 0x7f61a0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 --2- 172.21.15.16:0/541627660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61a01595b0 0x7f61a00b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 >> 172.21.15.16:0/541627660 conn(0x7f61a0099b40 msgr2=0x7f61a006f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 shutdown_connections 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.316+0000 7f61a6505700 1 -- 172.21.15.16:0/541627660 wait complete. 2023-10-06T11:42:05.322 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:05.330 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:42:04.061745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381212,"kb_used_data":2908,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593508,"statfs":{"total":773094113280,"available":770655752192,"internally_reserved":0,"allocated":2977792,"data_stored":1816046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002058"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517366+0000","last_change":"2023-10-06T11:41:56.510681+0000","last_active":"2023-10-06T11:41:56.517366+0000","last_peered":"2023-10-06T11:41:56.517366+0000","last_clean":"2023-10-06T11:41:56.517366+0000","last_became_active":"2023-10-06T11:41:56.510274+0000","last_became_peered":"2023-10-06T11:41:56.510274+0000","last_unstale":"2023-10-06T11:41:56.517366+0000","last_undegraded":"2023-10-06T11:41:56.517366+0000","last_fullsized":"2023-10-06T11:41:56.517366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:25.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519077+0000","last_change":"2023-10-06T11:41:56.512482+0000","last_active":"2023-10-06T11:41:56.519077+0000","last_peered":"2023-10-06T11:41:56.519077+0000","last_clean":"2023-10-06T11:41:56.519077+0000","last_became_active":"2023-10-06T11:41:56.512222+0000","last_became_peered":"2023-10-06T11:41:56.512222+0000","last_unstale":"2023-10-06T11:41:56.519077+0000","last_undegraded":"2023-10-06T11:41:56.519077+0000","last_fullsized":"2023-10-06T11:41:56.519077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513731+0000","last_change":"2023-10-06T11:41:56.509931+0000","last_active":"2023-10-06T11:41:56.513731+0000","last_peered":"2023-10-06T11:41:56.513731+0000","last_clean":"2023-10-06T11:41:56.513731+0000","last_became_active":"2023-10-06T11:41:56.509393+0000","last_became_peered":"2023-10-06T11:41:56.509393+0000","last_unstale":"2023-10-06T11:41:56.513731+0000","last_undegraded":"2023-10-06T11:41:56.513731+0000","last_fullsized":"2023-10-06T11:41:56.513731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:56.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516425+0000","last_change":"2023-10-06T11:41:56.511208+0000","last_active":"2023-10-06T11:41:56.516425+0000","last_peered":"2023-10-06T11:41:56.516425+0000","last_clean":"2023-10-06T11:41:56.516425+0000","last_became_active":"2023-10-06T11:41:56.510934+0000","last_became_peered":"2023-10-06T11:41:56.510934+0000","last_unstale":"2023-10-06T11:41:56.516425+0000","last_undegraded":"2023-10-06T11:41:56.516425+0000","last_fullsized":"2023-10-06T11:41:56.516425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:21.656559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519759+0000","last_change":"2023-10-06T11:41:56.516474+0000","last_active":"2023-10-06T11:41:56.519759+0000","last_peered":"2023-10-06T11:41:56.519759+0000","last_clean":"2023-10-06T11:41:56.519759+0000","last_became_active":"2023-10-06T11:41:56.516168+0000","last_became_peered":"2023-10-06T11:41:56.516168+0000","last_unstale":"2023-10-06T11:41:56.519759+0000","last_undegraded":"2023-10-06T11:41:56.519759+0000","last_fullsized":"2023-10-06T11:41:56.519759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513822+0000","last_change":"2023-10-06T11:41:56.509889+0000","last_active":"2023-10-06T11:41:56.513822+0000","last_peered":"2023-10-06T11:41:56.513822+0000","last_clean":"2023-10-06T11:41:56.513822+0000","last_became_active":"2023-10-06T11:41:56.509181+0000","last_became_peered":"2023-10-06T11:41:56.509181+0000","last_unstale":"2023-10-06T11:41:56.513822+0000","last_undegraded":"2023-10-06T11:41:56.513822+0000","last_fullsized":"2023-10-06T11:41:56.513822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:42.296263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522246+0000","last_change":"2023-10-06T11:41:56.518850+0000","last_active":"2023-10-06T11:41:56.522246+0000","last_peered":"2023-10-06T11:41:56.522246+0000","last_clean":"2023-10-06T11:41:56.522246+0000","last_became_active":"2023-10-06T11:41:56.518578+0000","last_became_peered":"2023-10-06T11:41:56.518578+0000","last_unstale":"2023-10-06T11:41:56.522246+0000","last_undegraded":"2023-10-06T11:41:56.522246+0000","last_fullsized":"2023-10-06T11:41:56.522246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514003+0000","last_change":"2023-10-06T11:41:56.508470+0000","last_active":"2023-10-06T11:41:56.514003+0000","last_peered":"2023-10-06T11:41:56.514003+0000","last_clean":"2023-10-06T11:41:56.514003+0000","last_became_active":"2023-10-06T11:41:56.508167+0000","last_became_peered":"2023-10-06T11:41:56.508167+0000","last_unstale":"2023-10-06T11:41:56.514003+0000","last_undegraded":"2023-10-06T11:41:56.514003+0000","last_fullsized":"2023-10-06T11:41:56.514003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519630+0000","last_change":"2023-10-06T11:41:56.517434+0000","last_active":"2023-10-06T11:41:56.519630+0000","last_peered":"2023-10-06T11:41:56.519630+0000","last_clean":"2023-10-06T11:41:56.519630+0000","last_became_active":"2023-10-06T11:41:56.517143+0000","last_became_peered":"2023-10-06T11:41:56.517143+0000","last_unstale":"2023-10-06T11:41:56.519630+0000","last_undegraded":"2023-10-06T11:41:56.519630+0000","last_fullsized":"2023-10-06T11:41:56.519630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514049+0000","last_change":"2023-10-06T11:41:56.510953+0000","last_active":"2023-10-06T11:41:56.514049+0000","last_peered":"2023-10-06T11:41:56.514049+0000","last_clean":"2023-10-06T11:41:56.514049+0000","last_became_active":"2023-10-06T11:41:56.510684+0000","last_became_peered":"2023-10-06T11:41:56.510684+0000","last_unstale":"2023-10-06T11:41:56.514049+0000","last_undegraded":"2023-10-06T11:41:56.514049+0000","last_fullsized":"2023-10-06T11:41:56.514049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516780+0000","last_change":"2023-10-06T11:41:56.511114+0000","last_active":"2023-10-06T11:41:56.516780+0000","last_peered":"2023-10-06T11:41:56.516780+0000","last_clean":"2023-10-06T11:41:56.516780+0000","last_became_active":"2023-10-06T11:41:56.510653+0000","last_became_peered":"2023-10-06T11:41:56.510653+0000","last_unstale":"2023-10-06T11:41:56.516780+0000","last_undegraded":"2023-10-06T11:41:56.516780+0000","last_fullsized":"2023-10-06T11:41:56.516780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518118+0000","last_change":"2023-10-06T11:41:56.512980+0000","last_active":"2023-10-06T11:41:56.518118+0000","last_peered":"2023-10-06T11:41:56.518118+0000","last_clean":"2023-10-06T11:41:56.518118+0000","last_became_active":"2023-10-06T11:41:56.512699+0000","last_became_peered":"2023-10-06T11:41:56.512699+0000","last_unstale":"2023-10-06T11:41:56.518118+0000","last_undegraded":"2023-10-06T11:41:56.518118+0000","last_fullsized":"2023-10-06T11:41:56.518118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:20.998002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518013+0000","last_change":"2023-10-06T11:41:56.512383+0000","last_active":"2023-10-06T11:41:56.518013+0000","last_peered":"2023-10-06T11:41:56.518013+0000","last_clean":"2023-10-06T11:41:56.518013+0000","last_became_active":"2023-10-06T11:41:56.512086+0000","last_became_peered":"2023-10-06T11:41:56.512086+0000","last_unstale":"2023-10-06T11:41:56.518013+0000","last_undegraded":"2023-10-06T11:41:56.518013+0000","last_fullsized":"2023-10-06T11:41:56.518013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:44.564348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514263+0000","last_change":"2023-10-06T11:41:56.510312+0000","last_active":"2023-10-06T11:41:56.514263+0000","last_peered":"2023-10-06T11:41:56.514263+0000","last_clean":"2023-10-06T11:41:56.514263+0000","last_became_active":"2023-10-06T11:41:56.510046+0000","last_became_peered":"2023-10-06T11:41:56.510046+0000","last_unstale":"2023-10-06T11:41:56.514263+0000","last_undegraded":"2023-10-06T11:41:56.514263+0000","last_fullsized":"2023-10-06T11:41:56.514263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522268+0000","last_change":"2023-10-06T11:41:56.518929+0000","last_active":"2023-10-06T11:41:56.522268+0000","last_peered":"2023-10-06T11:41:56.522268+0000","last_clean":"2023-10-06T11:41:56.522268+0000","last_became_active":"2023-10-06T11:41:56.518671+0000","last_became_peered":"2023-10-06T11:41:56.518671+0000","last_unstale":"2023-10-06T11:41:56.522268+0000","last_undegraded":"2023-10-06T11:41:56.522268+0000","last_fullsized":"2023-10-06T11:41:56.522268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513758+0000","last_change":"2023-10-06T11:41:56.509452+0000","last_active":"2023-10-06T11:41:56.513758+0000","last_peered":"2023-10-06T11:41:56.513758+0000","last_clean":"2023-10-06T11:41:56.513758+0000","last_became_active":"2023-10-06T11:41:56.509141+0000","last_became_peered":"2023-10-06T11:41:56.509141+0000","last_unstale":"2023-10-06T11:41:56.513758+0000","last_undegraded":"2023-10-06T11:41:56.513758+0000","last_fullsized":"2023-10-06T11:41:56.513758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515869+0000","last_change":"2023-10-06T11:41:56.511167+0000","last_active":"2023-10-06T11:41:56.515869+0000","last_peered":"2023-10-06T11:41:56.515869+0000","last_clean":"2023-10-06T11:41:56.515869+0000","last_became_active":"2023-10-06T11:41:56.510849+0000","last_became_peered":"2023-10-06T11:41:56.510849+0000","last_unstale":"2023-10-06T11:41:56.515869+0000","last_undegraded":"2023-10-06T11:41:56.515869+0000","last_fullsized":"2023-10-06T11:41:56.515869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516793+0000","last_change":"2023-10-06T11:41:56.511248+0000","last_active":"2023-10-06T11:41:56.516793+0000","last_peered":"2023-10-06T11:41:56.516793+0000","last_clean":"2023-10-06T11:41:56.516793+0000","last_became_active":"2023-10-06T11:41:56.510023+0000","last_became_peered":"2023-10-06T11:41:56.510023+0000","last_unstale":"2023-10-06T11:41:56.516793+0000","last_undegraded":"2023-10-06T11:41:56.516793+0000","last_fullsized":"2023-10-06T11:41:56.516793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519577+0000","last_change":"2023-10-06T11:41:56.513818+0000","last_active":"2023-10-06T11:41:56.519577+0000","last_peered":"2023-10-06T11:41:56.519577+0000","last_clean":"2023-10-06T11:41:56.519577+0000","last_became_active":"2023-10-06T11:41:56.513508+0000","last_became_peered":"2023-10-06T11:41:56.513508+0000","last_unstale":"2023-10-06T11:41:56.519577+0000","last_undegraded":"2023-10-06T11:41:56.519577+0000","last_fullsized":"2023-10-06T11:41:56.519577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:25.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519973+0000","last_change":"2023-10-06T11:41:56.514028+0000","last_active":"2023-10-06T11:41:56.519973+0000","last_peered":"2023-10-06T11:41:56.519973+0000","last_clean":"2023-10-06T11:41:56.519973+0000","last_became_active":"2023-10-06T11:41:56.513176+0000","last_became_peered":"2023-10-06T11:41:56.513176+0000","last_unstale":"2023-10-06T11:41:56.519973+0000","last_undegraded":"2023-10-06T11:41:56.519973+0000","last_fullsized":"2023-10-06T11:41:56.519973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516670+0000","last_change":"2023-10-06T11:41:56.513150+0000","last_active":"2023-10-06T11:41:56.516670+0000","last_peered":"2023-10-06T11:41:56.516670+0000","last_clean":"2023-10-06T11:41:56.516670+0000","last_became_active":"2023-10-06T11:41:56.512844+0000","last_became_peered":"2023-10-06T11:41:56.512844+0000","last_unstale":"2023-10-06T11:41:56.516670+0000","last_undegraded":"2023-10-06T11:41:56.516670+0000","last_fullsized":"2023-10-06T11:41:56.516670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519663+0000","last_change":"2023-10-06T11:41:56.512179+0000","last_active":"2023-10-06T11:41:56.519663+0000","last_peered":"2023-10-06T11:41:56.519663+0000","last_clean":"2023-10-06T11:41:56.519663+0000","last_became_active":"2023-10-06T11:41:56.511885+0000","last_became_peered":"2023-10-06T11:41:56.511885+0000","last_unstale":"2023-10-06T11:41:56.519663+0000","last_undegraded":"2023-10-06T11:41:56.519663+0000","last_fullsized":"2023-10-06T11:41:56.519663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520009+0000","last_change":"2023-10-06T11:41:56.514447+0000","last_active":"2023-10-06T11:41:56.520009+0000","last_peered":"2023-10-06T11:41:56.520009+0000","last_clean":"2023-10-06T11:41:56.520009+0000","last_became_active":"2023-10-06T11:41:56.514134+0000","last_became_peered":"2023-10-06T11:41:56.514134+0000","last_unstale":"2023-10-06T11:41:56.520009+0000","last_undegraded":"2023-10-06T11:41:56.520009+0000","last_fullsized":"2023-10-06T11:41:56.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516608+0000","last_change":"2023-10-06T11:41:56.513062+0000","last_active":"2023-10-06T11:41:56.516608+0000","last_peered":"2023-10-06T11:41:56.516608+0000","last_clean":"2023-10-06T11:41:56.516608+0000","last_became_active":"2023-10-06T11:41:56.512812+0000","last_became_peered":"2023-10-06T11:41:56.512812+0000","last_unstale":"2023-10-06T11:41:56.516608+0000","last_undegraded":"2023-10-06T11:41:56.516608+0000","last_fullsized":"2023-10-06T11:41:56.516608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517749+0000","last_change":"2023-10-06T11:41:56.514788+0000","last_active":"2023-10-06T11:41:56.517749+0000","last_peered":"2023-10-06T11:41:56.517749+0000","last_clean":"2023-10-06T11:41:56.517749+0000","last_became_active":"2023-10-06T11:41:56.514494+0000","last_became_peered":"2023-10-06T11:41:56.514494+0000","last_unstale":"2023-10-06T11:41:56.517749+0000","last_undegraded":"2023-10-06T11:41:56.517749+0000","last_fullsized":"2023-10-06T11:41:56.517749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516257+0000","last_change":"2023-10-06T11:41:56.511137+0000","last_active":"2023-10-06T11:41:56.516257+0000","last_peered":"2023-10-06T11:41:56.516257+0000","last_clean":"2023-10-06T11:41:56.516257+0000","last_became_active":"2023-10-06T11:41:56.510826+0000","last_became_peered":"2023-10-06T11:41:56.510826+0000","last_unstale":"2023-10-06T11:41:56.516257+0000","last_undegraded":"2023-10-06T11:41:56.516257+0000","last_fullsized":"2023-10-06T11:41:56.516257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519152+0000","last_change":"2023-10-06T11:41:56.513345+0000","last_active":"2023-10-06T11:41:56.519152+0000","last_peered":"2023-10-06T11:41:56.519152+0000","last_clean":"2023-10-06T11:41:56.519152+0000","last_became_active":"2023-10-06T11:41:56.513068+0000","last_became_peered":"2023-10-06T11:41:56.513068+0000","last_unstale":"2023-10-06T11:41:56.519152+0000","last_undegraded":"2023-10-06T11:41:56.519152+0000","last_fullsized":"2023-10-06T11:41:56.519152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.529014+0000","last_change":"2023-10-06T11:41:55.504888+0000","last_active":"2023-10-06T11:41:56.529014+0000","last_peered":"2023-10-06T11:41:56.529014+0000","last_clean":"2023-10-06T11:41:56.529014+0000","last_became_active":"2023-10-06T11:41:55.504602+0000","last_became_peered":"2023-10-06T11:41:55.504602+0000","last_unstale":"2023-10-06T11:41:56.529014+0000","last_undegraded":"2023-10-06T11:41:56.529014+0000","last_fullsized":"2023-10-06T11:41:56.529014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518587+0000","last_change":"2023-10-06T11:41:56.512500+0000","last_active":"2023-10-06T11:41:56.518587+0000","last_peered":"2023-10-06T11:41:56.518587+0000","last_clean":"2023-10-06T11:41:56.518587+0000","last_became_active":"2023-10-06T11:41:56.512213+0000","last_became_peered":"2023-10-06T11:41:56.512213+0000","last_unstale":"2023-10-06T11:41:56.518587+0000","last_undegraded":"2023-10-06T11:41:56.518587+0000","last_fullsized":"2023-10-06T11:41:56.518587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497430+0000","last_change":"2023-10-06T11:41:55.505880+0000","last_active":"2023-10-06T11:41:56.497430+0000","last_peered":"2023-10-06T11:41:56.497430+0000","last_clean":"2023-10-06T11:41:56.497430+0000","last_became_active":"2023-10-06T11:41:55.504907+0000","last_became_peered":"2023-10-06T11:41:55.504907+0000","last_unstale":"2023-10-06T11:41:56.497430+0000","last_undegraded":"2023-10-06T11:41:56.497430+0000","last_fullsized":"2023-10-06T11:41:56.497430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519692+0000","last_change":"2023-10-06T11:41:56.514094+0000","last_active":"2023-10-06T11:41:56.519692+0000","last_peered":"2023-10-06T11:41:56.519692+0000","last_clean":"2023-10-06T11:41:56.519692+0000","last_became_active":"2023-10-06T11:41:56.513833+0000","last_became_peered":"2023-10-06T11:41:56.513833+0000","last_unstale":"2023-10-06T11:41:56.519692+0000","last_undegraded":"2023-10-06T11:41:56.519692+0000","last_fullsized":"2023-10-06T11:41:56.519692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503895+0000","last_change":"2023-10-06T11:41:55.504864+0000","last_active":"2023-10-06T11:41:56.503895+0000","last_peered":"2023-10-06T11:41:56.503895+0000","last_clean":"2023-10-06T11:41:56.503895+0000","last_became_active":"2023-10-06T11:41:55.504166+0000","last_became_peered":"2023-10-06T11:41:55.504166+0000","last_unstale":"2023-10-06T11:41:56.503895+0000","last_undegraded":"2023-10-06T11:41:56.503895+0000","last_fullsized":"2023-10-06T11:41:56.503895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514214+0000","last_change":"2023-10-06T11:41:56.509697+0000","last_active":"2023-10-06T11:41:56.514214+0000","last_peered":"2023-10-06T11:41:56.514214+0000","last_clean":"2023-10-06T11:41:56.514214+0000","last_became_active":"2023-10-06T11:41:56.509414+0000","last_became_peered":"2023-10-06T11:41:56.509414+0000","last_unstale":"2023-10-06T11:41:56.514214+0000","last_undegraded":"2023-10-06T11:41:56.514214+0000","last_fullsized":"2023-10-06T11:41:56.514214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507789+0000","last_change":"2023-10-06T11:41:55.496678+0000","last_active":"2023-10-06T11:41:56.507789+0000","last_peered":"2023-10-06T11:41:56.507789+0000","last_clean":"2023-10-06T11:41:56.507789+0000","last_became_active":"2023-10-06T11:41:55.494896+0000","last_became_peered":"2023-10-06T11:41:55.494896+0000","last_unstale":"2023-10-06T11:41:56.507789+0000","last_undegraded":"2023-10-06T11:41:56.507789+0000","last_fullsized":"2023-10-06T11:41:56.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517092+0000","last_change":"2023-10-06T11:41:56.512202+0000","last_active":"2023-10-06T11:41:56.517092+0000","last_peered":"2023-10-06T11:41:56.517092+0000","last_clean":"2023-10-06T11:41:56.517092+0000","last_became_active":"2023-10-06T11:41:56.511913+0000","last_became_peered":"2023-10-06T11:41:56.511913+0000","last_unstale":"2023-10-06T11:41:56.517092+0000","last_undegraded":"2023-10-06T11:41:56.517092+0000","last_fullsized":"2023-10-06T11:41:56.517092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502121+0000","last_change":"2023-10-06T11:41:55.500940+0000","last_active":"2023-10-06T11:41:56.502121+0000","last_peered":"2023-10-06T11:41:56.502121+0000","last_clean":"2023-10-06T11:41:56.502121+0000","last_became_active":"2023-10-06T11:41:55.500652+0000","last_became_peered":"2023-10-06T11:41:55.500652+0000","last_unstale":"2023-10-06T11:41:56.502121+0000","last_undegraded":"2023-10-06T11:41:56.502121+0000","last_fullsized":"2023-10-06T11:41:56.502121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520515+0000","last_change":"2023-10-06T11:41:56.515837+0000","last_active":"2023-10-06T11:41:56.520515+0000","last_peered":"2023-10-06T11:41:56.520515+0000","last_clean":"2023-10-06T11:41:56.520515+0000","last_became_active":"2023-10-06T11:41:56.515511+0000","last_became_peered":"2023-10-06T11:41:56.515511+0000","last_unstale":"2023-10-06T11:41:56.520515+0000","last_undegraded":"2023-10-06T11:41:56.520515+0000","last_fullsized":"2023-10-06T11:41:56.520515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534860+0000","last_change":"2023-10-06T11:41:55.509167+0000","last_active":"2023-10-06T11:41:56.534860+0000","last_peered":"2023-10-06T11:41:56.534860+0000","last_clean":"2023-10-06T11:41:56.534860+0000","last_became_active":"2023-10-06T11:41:55.508968+0000","last_became_peered":"2023-10-06T11:41:55.508968+0000","last_unstale":"2023-10-06T11:41:56.534860+0000","last_undegraded":"2023-10-06T11:41:56.534860+0000","last_fullsized":"2023-10-06T11:41:56.534860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:13.549315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520107+0000","last_change":"2023-10-06T11:41:56.515835+0000","last_active":"2023-10-06T11:41:56.520107+0000","last_peered":"2023-10-06T11:41:56.520107+0000","last_clean":"2023-10-06T11:41:56.520107+0000","last_became_active":"2023-10-06T11:41:56.514755+0000","last_became_peered":"2023-10-06T11:41:56.514755+0000","last_unstale":"2023-10-06T11:41:56.520107+0000","last_undegraded":"2023-10-06T11:41:56.520107+0000","last_fullsized":"2023-10-06T11:41:56.520107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.536294+0000","last_change":"2023-10-06T11:41:55.503423+0000","last_active":"2023-10-06T11:41:56.536294+0000","last_peered":"2023-10-06T11:41:56.536294+0000","last_clean":"2023-10-06T11:41:56.536294+0000","last_became_active":"2023-10-06T11:41:55.499943+0000","last_became_peered":"2023-10-06T11:41:55.499943+0000","last_unstale":"2023-10-06T11:41:56.536294+0000","last_undegraded":"2023-10-06T11:41:56.536294+0000","last_fullsized":"2023-10-06T11:41:56.536294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519939+0000","last_change":"2023-10-06T11:41:56.513385+0000","last_active":"2023-10-06T11:41:56.519939+0000","last_peered":"2023-10-06T11:41:56.519939+0000","last_clean":"2023-10-06T11:41:56.519939+0000","last_became_active":"2023-10-06T11:41:56.513091+0000","last_became_peered":"2023-10-06T11:41:56.513091+0000","last_unstale":"2023-10-06T11:41:56.519939+0000","last_undegraded":"2023-10-06T11:41:56.519939+0000","last_fullsized":"2023-10-06T11:41:56.519939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:31.044745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534779+0000","last_change":"2023-10-06T11:41:55.508640+0000","last_active":"2023-10-06T11:41:56.534779+0000","last_peered":"2023-10-06T11:41:56.534779+0000","last_clean":"2023-10-06T11:41:56.534779+0000","last_became_active":"2023-10-06T11:41:55.508359+0000","last_became_peered":"2023-10-06T11:41:55.508359+0000","last_unstale":"2023-10-06T11:41:56.534779+0000","last_undegraded":"2023-10-06T11:41:56.534779+0000","last_fullsized":"2023-10-06T11:41:56.534779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:20.652268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520221+0000","last_change":"2023-10-06T11:41:56.518196+0000","last_active":"2023-10-06T11:41:56.520221+0000","last_peered":"2023-10-06T11:41:56.520221+0000","last_clean":"2023-10-06T11:41:56.520221+0000","last_became_active":"2023-10-06T11:41:56.517922+0000","last_became_peered":"2023-10-06T11:41:56.517922+0000","last_unstale":"2023-10-06T11:41:56.520221+0000","last_undegraded":"2023-10-06T11:41:56.520221+0000","last_fullsized":"2023-10-06T11:41:56.520221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507442+0000","last_change":"2023-10-06T11:41:55.498881+0000","last_active":"2023-10-06T11:41:56.507442+0000","last_peered":"2023-10-06T11:41:56.507442+0000","last_clean":"2023-10-06T11:41:56.507442+0000","last_became_active":"2023-10-06T11:41:55.498646+0000","last_became_peered":"2023-10-06T11:41:55.498646+0000","last_unstale":"2023-10-06T11:41:56.507442+0000","last_undegraded":"2023-10-06T11:41:56.507442+0000","last_fullsized":"2023-10-06T11:41:56.507442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.521939+0000","last_change":"2023-10-06T11:41:56.514291+0000","last_active":"2023-10-06T11:41:56.521939+0000","last_peered":"2023-10-06T11:41:56.521939+0000","last_clean":"2023-10-06T11:41:56.521939+0000","last_became_active":"2023-10-06T11:41:56.513811+0000","last_became_peered":"2023-10-06T11:41:56.513811+0000","last_unstale":"2023-10-06T11:41:56.521939+0000","last_undegraded":"2023-10-06T11:41:56.521939+0000","last_fullsized":"2023-10-06T11:41:56.521939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506360+0000","last_change":"2023-10-06T11:41:55.510456+0000","last_active":"2023-10-06T11:41:56.506360+0000","last_peered":"2023-10-06T11:41:56.506360+0000","last_clean":"2023-10-06T11:41:56.506360+0000","last_became_active":"2023-10-06T11:41:55.510148+0000","last_became_peered":"2023-10-06T11:41:55.510148+0000","last_unstale":"2023-10-06T11:41:56.506360+0000","last_undegraded":"2023-10-06T11:41:56.506360+0000","last_fullsized":"2023-10-06T11:41:56.506360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519708+0000","last_change":"2023-10-06T11:41:56.516787+0000","last_active":"2023-10-06T11:41:56.519708+0000","last_peered":"2023-10-06T11:41:56.519708+0000","last_clean":"2023-10-06T11:41:56.519708+0000","last_became_active":"2023-10-06T11:41:56.516510+0000","last_became_peered":"2023-10-06T11:41:56.516510+0000","last_unstale":"2023-10-06T11:41:56.519708+0000","last_undegraded":"2023-10-06T11:41:56.519708+0000","last_fullsized":"2023-10-06T11:41:56.519708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.532331+0000","last_change":"2023-10-06T11:41:55.502794+0000","last_active":"2023-10-06T11:41:56.532331+0000","last_peered":"2023-10-06T11:41:56.532331+0000","last_clean":"2023-10-06T11:41:56.532331+0000","last_became_active":"2023-10-06T11:41:55.502522+0000","last_became_peered":"2023-10-06T11:41:55.502522+0000","last_unstale":"2023-10-06T11:41:56.532331+0000","last_undegraded":"2023-10-06T11:41:56.532331+0000","last_fullsized":"2023-10-06T11:41:56.532331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520360+0000","last_change":"2023-10-06T11:41:56.515679+0000","last_active":"2023-10-06T11:41:56.520360+0000","last_peered":"2023-10-06T11:41:56.520360+0000","last_clean":"2023-10-06T11:41:56.520360+0000","last_became_active":"2023-10-06T11:41:56.515167+0000","last_became_peered":"2023-10-06T11:41:56.515167+0000","last_unstale":"2023-10-06T11:41:56.520360+0000","last_undegraded":"2023-10-06T11:41:56.520360+0000","last_fullsized":"2023-10-06T11:41:56.520360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534434+0000","last_change":"2023-10-06T11:41:55.506331+0000","last_active":"2023-10-06T11:41:56.534434+0000","last_peered":"2023-10-06T11:41:56.534434+0000","last_clean":"2023-10-06T11:41:56.534434+0000","last_became_active":"2023-10-06T11:41:55.506044+0000","last_became_peered":"2023-10-06T11:41:55.506044+0000","last_unstale":"2023-10-06T11:41:56.534434+0000","last_undegraded":"2023-10-06T11:41:56.534434+0000","last_fullsized":"2023-10-06T11:41:56.534434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514023+0000","last_change":"2023-10-06T11:41:56.509291+0000","last_active":"2023-10-06T11:41:56.514023+0000","last_peered":"2023-10-06T11:41:56.514023+0000","last_clean":"2023-10-06T11:41:56.514023+0000","last_became_active":"2023-10-06T11:41:56.509004+0000","last_became_peered":"2023-10-06T11:41:56.509004+0000","last_unstale":"2023-10-06T11:41:56.514023+0000","last_undegraded":"2023-10-06T11:41:56.514023+0000","last_fullsized":"2023-10-06T11:41:56.514023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534956+0000","last_change":"2023-10-06T11:41:55.510357+0000","last_active":"2023-10-06T11:41:56.534956+0000","last_peered":"2023-10-06T11:41:56.534956+0000","last_clean":"2023-10-06T11:41:56.534956+0000","last_became_active":"2023-10-06T11:41:55.510053+0000","last_became_peered":"2023-10-06T11:41:55.510053+0000","last_unstale":"2023-10-06T11:41:56.534956+0000","last_undegraded":"2023-10-06T11:41:56.534956+0000","last_fullsized":"2023-10-06T11:41:56.534956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516748+0000","last_change":"2023-10-06T11:41:56.510920+0000","last_active":"2023-10-06T11:41:56.516748+0000","last_peered":"2023-10-06T11:41:56.516748+0000","last_clean":"2023-10-06T11:41:56.516748+0000","last_became_active":"2023-10-06T11:41:56.510620+0000","last_became_peered":"2023-10-06T11:41:56.510620+0000","last_unstale":"2023-10-06T11:41:56.516748+0000","last_undegraded":"2023-10-06T11:41:56.516748+0000","last_fullsized":"2023-10-06T11:41:56.516748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:01.202344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534773+0000","last_change":"2023-10-06T11:41:55.501172+0000","last_active":"2023-10-06T11:41:56.534773+0000","last_peered":"2023-10-06T11:41:56.534773+0000","last_clean":"2023-10-06T11:41:56.534773+0000","last_became_active":"2023-10-06T11:41:55.500891+0000","last_became_peered":"2023-10-06T11:41:55.500891+0000","last_unstale":"2023-10-06T11:41:56.534773+0000","last_undegraded":"2023-10-06T11:41:56.534773+0000","last_fullsized":"2023-10-06T11:41:56.534773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:31.984593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516634+0000","last_change":"2023-10-06T11:41:56.512054+0000","last_active":"2023-10-06T11:41:56.516634+0000","last_peered":"2023-10-06T11:41:56.516634+0000","last_clean":"2023-10-06T11:41:56.516634+0000","last_became_active":"2023-10-06T11:41:56.511385+0000","last_became_peered":"2023-10-06T11:41:56.511385+0000","last_unstale":"2023-10-06T11:41:56.516634+0000","last_undegraded":"2023-10-06T11:41:56.516634+0000","last_fullsized":"2023-10-06T11:41:56.516634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.394264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.501453+0000","last_change":"2023-10-06T11:41:55.505446+0000","last_active":"2023-10-06T11:41:56.501453+0000","last_peered":"2023-10-06T11:41:56.501453+0000","last_clean":"2023-10-06T11:41:56.501453+0000","last_became_active":"2023-10-06T11:41:55.505189+0000","last_became_peered":"2023-10-06T11:41:55.505189+0000","last_unstale":"2023-10-06T11:41:56.501453+0000","last_undegraded":"2023-10-06T11:41:56.501453+0000","last_fullsized":"2023-10-06T11:41:56.501453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519776+0000","last_change":"2023-10-06T11:41:56.517521+0000","last_active":"2023-10-06T11:41:56.519776+0000","last_peered":"2023-10-06T11:41:56.519776+0000","last_clean":"2023-10-06T11:41:56.519776+0000","last_became_active":"2023-10-06T11:41:56.517084+0000","last_became_peered":"2023-10-06T11:41:56.517084+0000","last_unstale":"2023-10-06T11:41:56.519776+0000","last_undegraded":"2023-10-06T11:41:56.519776+0000","last_fullsized":"2023-10-06T11:41:56.519776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531354+0000","last_change":"2023-10-06T11:41:55.509060+0000","last_active":"2023-10-06T11:41:56.531354+0000","last_peered":"2023-10-06T11:41:56.531354+0000","last_clean":"2023-10-06T11:41:56.531354+0000","last_became_active":"2023-10-06T11:41:55.508766+0000","last_became_peered":"2023-10-06T11:41:55.508766+0000","last_unstale":"2023-10-06T11:41:56.531354+0000","last_undegraded":"2023-10-06T11:41:56.531354+0000","last_fullsized":"2023-10-06T11:41:56.531354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514111+0000","last_change":"2023-10-06T11:41:56.510155+0000","last_active":"2023-10-06T11:41:56.514111+0000","last_peered":"2023-10-06T11:41:56.514111+0000","last_clean":"2023-10-06T11:41:56.514111+0000","last_became_active":"2023-10-06T11:41:56.509859+0000","last_became_peered":"2023-10-06T11:41:56.509859+0000","last_unstale":"2023-10-06T11:41:56.514111+0000","last_undegraded":"2023-10-06T11:41:56.514111+0000","last_fullsized":"2023-10-06T11:41:56.514111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507800+0000","last_change":"2023-10-06T11:41:55.511135+0000","last_active":"2023-10-06T11:41:56.507800+0000","last_peered":"2023-10-06T11:41:56.507800+0000","last_clean":"2023-10-06T11:41:56.507800+0000","last_became_active":"2023-10-06T11:41:55.510802+0000","last_became_peered":"2023-10-06T11:41:55.510802+0000","last_unstale":"2023-10-06T11:41:56.507800+0000","last_undegraded":"2023-10-06T11:41:56.507800+0000","last_fullsized":"2023-10-06T11:41:56.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516764+0000","last_change":"2023-10-06T11:41:56.510930+0000","last_active":"2023-10-06T11:41:56.516764+0000","last_peered":"2023-10-06T11:41:56.516764+0000","last_clean":"2023-10-06T11:41:56.516764+0000","last_became_active":"2023-10-06T11:41:56.510315+0000","last_became_peered":"2023-10-06T11:41:56.510315+0000","last_unstale":"2023-10-06T11:41:56.516764+0000","last_undegraded":"2023-10-06T11:41:56.516764+0000","last_fullsized":"2023-10-06T11:41:56.516764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500145+0000","last_change":"2023-10-06T11:41:55.506086+0000","last_active":"2023-10-06T11:41:56.500145+0000","last_peered":"2023-10-06T11:41:56.500145+0000","last_clean":"2023-10-06T11:41:56.500145+0000","last_became_active":"2023-10-06T11:41:55.505822+0000","last_became_peered":"2023-10-06T11:41:55.505822+0000","last_unstale":"2023-10-06T11:41:56.500145+0000","last_undegraded":"2023-10-06T11:41:56.500145+0000","last_fullsized":"2023-10-06T11:41:56.500145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520047+0000","last_change":"2023-10-06T11:41:56.514492+0000","last_active":"2023-10-06T11:41:56.520047+0000","last_peered":"2023-10-06T11:41:56.520047+0000","last_clean":"2023-10-06T11:41:56.520047+0000","last_became_active":"2023-10-06T11:41:56.513778+0000","last_became_peered":"2023-10-06T11:41:56.513778+0000","last_unstale":"2023-10-06T11:41:56.520047+0000","last_undegraded":"2023-10-06T11:41:56.520047+0000","last_fullsized":"2023-10-06T11:41:56.520047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503812+0000","last_change":"2023-10-06T11:41:55.505373+0000","last_active":"2023-10-06T11:41:56.503812+0000","last_peered":"2023-10-06T11:41:56.503812+0000","last_clean":"2023-10-06T11:41:56.503812+0000","last_became_active":"2023-10-06T11:41:55.505092+0000","last_became_peered":"2023-10-06T11:41:55.505092+0000","last_unstale":"2023-10-06T11:41:56.503812+0000","last_undegraded":"2023-10-06T11:41:56.503812+0000","last_fullsized":"2023-10-06T11:41:56.503812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516491+0000","last_change":"2023-10-06T11:41:56.512965+0000","last_active":"2023-10-06T11:41:56.516491+0000","last_peered":"2023-10-06T11:41:56.516491+0000","last_clean":"2023-10-06T11:41:56.516491+0000","last_became_active":"2023-10-06T11:41:56.512684+0000","last_became_peered":"2023-10-06T11:41:56.512684+0000","last_unstale":"2023-10-06T11:41:56.516491+0000","last_undegraded":"2023-10-06T11:41:56.516491+0000","last_fullsized":"2023-10-06T11:41:56.516491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534650+0000","last_change":"2023-10-06T11:41:55.510312+0000","last_active":"2023-10-06T11:41:56.534650+0000","last_peered":"2023-10-06T11:41:56.534650+0000","last_clean":"2023-10-06T11:41:56.534650+0000","last_became_active":"2023-10-06T11:41:55.510044+0000","last_became_peered":"2023-10-06T11:41:55.510044+0000","last_unstale":"2023-10-06T11:41:56.534650+0000","last_undegraded":"2023-10-06T11:41:56.534650+0000","last_fullsized":"2023-10-06T11:41:56.534650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500088+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:41:56.500088+0000","last_peered":"2023-10-06T11:41:56.500088+0000","last_clean":"2023-10-06T11:41:56.500088+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:41:56.500088+0000","last_undegraded":"2023-10-06T11:41:56.500088+0000","last_fullsized":"2023-10-06T11:41:56.500088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519625+0000","last_change":"2023-10-06T11:41:56.515565+0000","last_active":"2023-10-06T11:41:56.519625+0000","last_peered":"2023-10-06T11:41:56.519625+0000","last_clean":"2023-10-06T11:41:56.519625+0000","last_became_active":"2023-10-06T11:41:56.515007+0000","last_became_peered":"2023-10-06T11:41:56.515007+0000","last_unstale":"2023-10-06T11:41:56.519625+0000","last_undegraded":"2023-10-06T11:41:56.519625+0000","last_fullsized":"2023-10-06T11:41:56.519625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503231+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:41:56.503231+0000","last_peered":"2023-10-06T11:41:56.503231+0000","last_clean":"2023-10-06T11:41:56.503231+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:41:56.503231+0000","last_undegraded":"2023-10-06T11:41:56.503231+0000","last_fullsized":"2023-10-06T11:41:56.503231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:41:58.260002+0000","last_change":"2023-10-06T11:41:55.500074+0000","last_active":"2023-10-06T11:41:58.260002+0000","last_peered":"2023-10-06T11:41:58.260002+0000","last_clean":"2023-10-06T11:41:58.260002+0000","last_became_active":"2023-10-06T11:41:55.499784+0000","last_became_peered":"2023-10-06T11:41:55.499784+0000","last_unstale":"2023-10-06T11:41:58.260002+0000","last_undegraded":"2023-10-06T11:41:58.260002+0000","last_fullsized":"2023-10-06T11:41:58.260002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513641+0000","last_change":"2023-10-06T11:41:56.509030+0000","last_active":"2023-10-06T11:41:56.513641+0000","last_peered":"2023-10-06T11:41:56.513641+0000","last_clean":"2023-10-06T11:41:56.513641+0000","last_became_active":"2023-10-06T11:41:56.508720+0000","last_became_peered":"2023-10-06T11:41:56.508720+0000","last_unstale":"2023-10-06T11:41:56.513641+0000","last_undegraded":"2023-10-06T11:41:56.513641+0000","last_fullsized":"2023-10-06T11:41:56.513641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:51.155604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497934+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:41:56.497934+0000","last_peered":"2023-10-06T11:41:56.497934+0000","last_clean":"2023-10-06T11:41:56.497934+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:41:56.497934+0000","last_undegraded":"2023-10-06T11:41:56.497934+0000","last_fullsized":"2023-10-06T11:41:56.497934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506182+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:41:56.506182+0000","last_peered":"2023-10-06T11:41:56.506182+0000","last_clean":"2023-10-06T11:41:56.506182+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:41:56.506182+0000","last_undegraded":"2023-10-06T11:41:56.506182+0000","last_fullsized":"2023-10-06T11:41:56.506182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507640+0000","last_change":"2023-10-06T11:41:55.506608+0000","last_active":"2023-10-06T11:41:56.507640+0000","last_peered":"2023-10-06T11:41:56.507640+0000","last_clean":"2023-10-06T11:41:56.507640+0000","last_became_active":"2023-10-06T11:41:55.506234+0000","last_became_peered":"2023-10-06T11:41:55.506234+0000","last_unstale":"2023-10-06T11:41:56.507640+0000","last_undegraded":"2023-10-06T11:41:56.507640+0000","last_fullsized":"2023-10-06T11:41:56.507640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519724+0000","last_change":"2023-10-06T11:41:56.516915+0000","last_active":"2023-10-06T11:41:56.519724+0000","last_peered":"2023-10-06T11:41:56.519724+0000","last_clean":"2023-10-06T11:41:56.519724+0000","last_became_active":"2023-10-06T11:41:56.516184+0000","last_became_peered":"2023-10-06T11:41:56.516184+0000","last_unstale":"2023-10-06T11:41:56.519724+0000","last_undegraded":"2023-10-06T11:41:56.519724+0000","last_fullsized":"2023-10-06T11:41:56.519724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505779+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:41:56.505779+0000","last_peered":"2023-10-06T11:41:56.505779+0000","last_clean":"2023-10-06T11:41:56.505779+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:41:56.505779+0000","last_undegraded":"2023-10-06T11:41:56.505779+0000","last_fullsized":"2023-10-06T11:41:56.505779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497472+0000","last_change":"2023-10-06T11:41:55.506060+0000","last_active":"2023-10-06T11:41:56.497472+0000","last_peered":"2023-10-06T11:41:56.497472+0000","last_clean":"2023-10-06T11:41:56.497472+0000","last_became_active":"2023-10-06T11:41:55.505551+0000","last_became_peered":"2023-10-06T11:41:55.505551+0000","last_unstale":"2023-10-06T11:41:56.497472+0000","last_undegraded":"2023-10-06T11:41:56.497472+0000","last_fullsized":"2023-10-06T11:41:56.497472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:23.340598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515790+0000","last_change":"2023-10-06T11:41:56.509141+0000","last_active":"2023-10-06T11:41:56.515790+0000","last_peered":"2023-10-06T11:41:56.515790+0000","last_clean":"2023-10-06T11:41:56.515790+0000","last_became_active":"2023-10-06T11:41:56.508814+0000","last_became_peered":"2023-10-06T11:41:56.508814+0000","last_unstale":"2023-10-06T11:41:56.515790+0000","last_undegraded":"2023-10-06T11:41:56.515790+0000","last_fullsized":"2023-10-06T11:41:56.515790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505943+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:41:56.505943+0000","last_peered":"2023-10-06T11:41:56.505943+0000","last_clean":"2023-10-06T11:41:56.505943+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:41:56.505943+0000","last_undegraded":"2023-10-06T11:41:56.505943+0000","last_fullsized":"2023-10-06T11:41:56.505943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502767+0000","last_change":"2023-10-06T11:41:55.504016+0000","last_active":"2023-10-06T11:41:56.502767+0000","last_peered":"2023-10-06T11:41:56.502767+0000","last_clean":"2023-10-06T11:41:56.502767+0000","last_became_active":"2023-10-06T11:41:55.503831+0000","last_became_peered":"2023-10-06T11:41:55.503831+0000","last_unstale":"2023-10-06T11:41:56.502767+0000","last_undegraded":"2023-10-06T11:41:56.502767+0000","last_fullsized":"2023-10-06T11:41:56.502767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519487+0000","last_change":"2023-10-06T11:41:56.515759+0000","last_active":"2023-10-06T11:41:56.519487+0000","last_peered":"2023-10-06T11:41:56.519487+0000","last_clean":"2023-10-06T11:41:56.519487+0000","last_became_active":"2023-10-06T11:41:56.514780+0000","last_became_peered":"2023-10-06T11:41:56.514780+0000","last_unstale":"2023-10-06T11:41:56.519487+0000","last_undegraded":"2023-10-06T11:41:56.519487+0000","last_fullsized":"2023-10-06T11:41:56.519487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502819+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:41:56.502819+0000","last_peered":"2023-10-06T11:41:56.502819+0000","last_clean":"2023-10-06T11:41:56.502819+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:41:56.502819+0000","last_undegraded":"2023-10-06T11:41:56.502819+0000","last_fullsized":"2023-10-06T11:41:56.502819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500296+0000","last_change":"2023-10-06T11:41:55.508163+0000","last_active":"2023-10-06T11:41:56.500296+0000","last_peered":"2023-10-06T11:41:56.500296+0000","last_clean":"2023-10-06T11:41:56.500296+0000","last_became_active":"2023-10-06T11:41:55.507855+0000","last_became_peered":"2023-10-06T11:41:55.507855+0000","last_unstale":"2023-10-06T11:41:56.500296+0000","last_undegraded":"2023-10-06T11:41:56.500296+0000","last_fullsized":"2023-10-06T11:41:56.500296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516585+0000","last_change":"2023-10-06T11:41:56.512837+0000","last_active":"2023-10-06T11:41:56.516585+0000","last_peered":"2023-10-06T11:41:56.516585+0000","last_clean":"2023-10-06T11:41:56.516585+0000","last_became_active":"2023-10-06T11:41:56.511727+0000","last_became_peered":"2023-10-06T11:41:56.511727+0000","last_unstale":"2023-10-06T11:41:56.516585+0000","last_undegraded":"2023-10-06T11:41:56.516585+0000","last_fullsized":"2023-10-06T11:41:56.516585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507674+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:41:56.507674+0000","last_peered":"2023-10-06T11:41:56.507674+0000","last_clean":"2023-10-06T11:41:56.507674+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:41:56.507674+0000","last_undegraded":"2023-10-06T11:41:56.507674+0000","last_fullsized":"2023-10-06T11:41:56.507674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497703+0000","last_change":"2023-10-06T11:41:55.508524+0000","last_active":"2023-10-06T11:41:56.497703+0000","last_peered":"2023-10-06T11:41:56.497703+0000","last_clean":"2023-10-06T11:41:56.497703+0000","last_became_active":"2023-10-06T11:41:55.505590+0000","last_became_peered":"2023-10-06T11:41:55.505590+0000","last_unstale":"2023-10-06T11:41:56.497703+0000","last_undegraded":"2023-10-06T11:41:56.497703+0000","last_fullsized":"2023-10-06T11:41:56.497703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.511544+0000","last_change":"2023-10-06T11:41:56.509540+0000","last_active":"2023-10-06T11:41:56.511544+0000","last_peered":"2023-10-06T11:41:56.511544+0000","last_clean":"2023-10-06T11:41:56.511544+0000","last_became_active":"2023-10-06T11:41:56.509208+0000","last_became_peered":"2023-10-06T11:41:56.509208+0000","last_unstale":"2023-10-06T11:41:56.511544+0000","last_undegraded":"2023-10-06T11:41:56.511544+0000","last_fullsized":"2023-10-06T11:41:56.511544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507693+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:41:56.507693+0000","last_peered":"2023-10-06T11:41:56.507693+0000","last_clean":"2023-10-06T11:41:56.507693+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:41:56.507693+0000","last_undegraded":"2023-10-06T11:41:56.507693+0000","last_fullsized":"2023-10-06T11:41:56.507693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503179+0000","last_change":"2023-10-06T11:41:55.505392+0000","last_active":"2023-10-06T11:41:56.503179+0000","last_peered":"2023-10-06T11:41:56.503179+0000","last_clean":"2023-10-06T11:41:56.503179+0000","last_became_active":"2023-10-06T11:41:55.505103+0000","last_became_peered":"2023-10-06T11:41:55.505103+0000","last_unstale":"2023-10-06T11:41:56.503179+0000","last_undegraded":"2023-10-06T11:41:56.503179+0000","last_fullsized":"2023-10-06T11:41:56.503179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516182+0000","last_change":"2023-10-06T11:41:56.509787+0000","last_active":"2023-10-06T11:41:56.516182+0000","last_peered":"2023-10-06T11:41:56.516182+0000","last_clean":"2023-10-06T11:41:56.516182+0000","last_became_active":"2023-10-06T11:41:56.509495+0000","last_became_peered":"2023-10-06T11:41:56.509495+0000","last_unstale":"2023-10-06T11:41:56.516182+0000","last_undegraded":"2023-10-06T11:41:56.516182+0000","last_fullsized":"2023-10-06T11:41:56.516182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531881+0000","last_change":"2023-10-06T11:41:55.501475+0000","last_active":"2023-10-06T11:41:56.531881+0000","last_peered":"2023-10-06T11:41:56.531881+0000","last_clean":"2023-10-06T11:41:56.531881+0000","last_became_active":"2023-10-06T11:41:55.500531+0000","last_became_peered":"2023-10-06T11:41:55.500531+0000","last_unstale":"2023-10-06T11:41:56.531881+0000","last_undegraded":"2023-10-06T11:41:56.531881+0000","last_fullsized":"2023-10-06T11:41:56.531881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519814+0000","last_change":"2023-10-06T11:41:56.518985+0000","last_active":"2023-10-06T11:41:56.519814+0000","last_peered":"2023-10-06T11:41:56.519814+0000","last_clean":"2023-10-06T11:41:56.519814+0000","last_became_active":"2023-10-06T11:41:56.518656+0000","last_became_peered":"2023-10-06T11:41:56.518656+0000","last_unstale":"2023-10-06T11:41:56.519814+0000","last_undegraded":"2023-10-06T11:41:56.519814+0000","last_fullsized":"2023-10-06T11:41:56.519814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:21.108193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534877+0000","last_change":"2023-10-06T11:41:55.506318+0000","last_active":"2023-10-06T11:41:56.534877+0000","last_peered":"2023-10-06T11:41:56.534877+0000","last_clean":"2023-10-06T11:41:56.534877+0000","last_became_active":"2023-10-06T11:41:55.506045+0000","last_became_peered":"2023-10-06T11:41:55.506045+0000","last_unstale":"2023-10-06T11:41:56.534877+0000","last_undegraded":"2023-10-06T11:41:56.534877+0000","last_fullsized":"2023-10-06T11:41:56.534877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520066+0000","last_change":"2023-10-06T11:41:56.515883+0000","last_active":"2023-10-06T11:41:56.520066+0000","last_peered":"2023-10-06T11:41:56.520066+0000","last_clean":"2023-10-06T11:41:56.520066+0000","last_became_active":"2023-10-06T11:41:56.515622+0000","last_became_peered":"2023-10-06T11:41:56.515622+0000","last_unstale":"2023-10-06T11:41:56.520066+0000","last_undegraded":"2023-10-06T11:41:56.520066+0000","last_fullsized":"2023-10-06T11:41:56.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:31.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.535054+0000","last_change":"2023-10-06T11:41:55.507981+0000","last_active":"2023-10-06T11:41:56.535054+0000","last_peered":"2023-10-06T11:41:56.535054+0000","last_clean":"2023-10-06T11:41:56.535054+0000","last_became_active":"2023-10-06T11:41:55.507665+0000","last_became_peered":"2023-10-06T11:41:55.507665+0000","last_unstale":"2023-10-06T11:41:56.535054+0000","last_undegraded":"2023-10-06T11:41:56.535054+0000","last_fullsized":"2023-10-06T11:41:56.535054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513856+0000","last_change":"2023-10-06T11:41:56.510088+0000","last_active":"2023-10-06T11:41:56.513856+0000","last_peered":"2023-10-06T11:41:56.513856+0000","last_clean":"2023-10-06T11:41:56.513856+0000","last_became_active":"2023-10-06T11:41:56.509600+0000","last_became_peered":"2023-10-06T11:41:56.509600+0000","last_unstale":"2023-10-06T11:41:56.513856+0000","last_undegraded":"2023-10-06T11:41:56.513856+0000","last_fullsized":"2023-10-06T11:41:56.513856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:16.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534276+0000","last_change":"2023-10-06T11:41:55.503624+0000","last_active":"2023-10-06T11:41:56.534276+0000","last_peered":"2023-10-06T11:41:56.534276+0000","last_clean":"2023-10-06T11:41:56.534276+0000","last_became_active":"2023-10-06T11:41:55.503318+0000","last_became_peered":"2023-10-06T11:41:55.503318+0000","last_unstale":"2023-10-06T11:41:56.534276+0000","last_undegraded":"2023-10-06T11:41:56.534276+0000","last_fullsized":"2023-10-06T11:41:56.534276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:44.190608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519991+0000","last_change":"2023-10-06T11:41:56.510835+0000","last_active":"2023-10-06T11:41:56.519991+0000","last_peered":"2023-10-06T11:41:56.519991+0000","last_clean":"2023-10-06T11:41:56.519991+0000","last_became_active":"2023-10-06T11:41:56.510547+0000","last_became_peered":"2023-10-06T11:41:56.510547+0000","last_unstale":"2023-10-06T11:41:56.519991+0000","last_undegraded":"2023-10-06T11:41:56.519991+0000","last_fullsized":"2023-10-06T11:41:56.519991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534863+0000","last_change":"2023-10-06T11:41:55.497178+0000","last_active":"2023-10-06T11:41:56.534863+0000","last_peered":"2023-10-06T11:41:56.534863+0000","last_clean":"2023-10-06T11:41:56.534863+0000","last_became_active":"2023-10-06T11:41:55.495336+0000","last_became_peered":"2023-10-06T11:41:55.495336+0000","last_unstale":"2023-10-06T11:41:56.534863+0000","last_undegraded":"2023-10-06T11:41:56.534863+0000","last_fullsized":"2023-10-06T11:41:56.534863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519888+0000","last_change":"2023-10-06T11:41:56.511788+0000","last_active":"2023-10-06T11:41:56.519888+0000","last_peered":"2023-10-06T11:41:56.519888+0000","last_clean":"2023-10-06T11:41:56.519888+0000","last_became_active":"2023-10-06T11:41:56.511488+0000","last_became_peered":"2023-10-06T11:41:56.511488+0000","last_unstale":"2023-10-06T11:41:56.519888+0000","last_undegraded":"2023-10-06T11:41:56.519888+0000","last_fullsized":"2023-10-06T11:41:56.519888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516639+0000","last_change":"2023-10-06T11:41:56.511919+0000","last_active":"2023-10-06T11:41:56.516639+0000","last_peered":"2023-10-06T11:41:56.516639+0000","last_clean":"2023-10-06T11:41:56.516639+0000","last_became_active":"2023-10-06T11:41:56.511644+0000","last_became_peered":"2023-10-06T11:41:56.511644+0000","last_unstale":"2023-10-06T11:41:56.516639+0000","last_undegraded":"2023-10-06T11:41:56.516639+0000","last_fullsized":"2023-10-06T11:41:56.516639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520154+0000","last_change":"2023-10-06T11:41:56.516583+0000","last_active":"2023-10-06T11:41:56.520154+0000","last_peered":"2023-10-06T11:41:56.520154+0000","last_clean":"2023-10-06T11:41:56.520154+0000","last_became_active":"2023-10-06T11:41:56.516241+0000","last_became_peered":"2023-10-06T11:41:56.516241+0000","last_unstale":"2023-10-06T11:41:56.520154+0000","last_undegraded":"2023-10-06T11:41:56.520154+0000","last_fullsized":"2023-10-06T11:41:56.520154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520023+0000","last_change":"2023-10-06T11:41:56.514199+0000","last_active":"2023-10-06T11:41:56.520023+0000","last_peered":"2023-10-06T11:41:56.520023+0000","last_clean":"2023-10-06T11:41:56.520023+0000","last_became_active":"2023-10-06T11:41:56.513961+0000","last_became_peered":"2023-10-06T11:41:56.513961+0000","last_unstale":"2023-10-06T11:41:56.520023+0000","last_undegraded":"2023-10-06T11:41:56.520023+0000","last_fullsized":"2023-10-06T11:41:56.520023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520267+0000","last_change":"2023-10-06T11:41:56.518788+0000","last_active":"2023-10-06T11:41:56.520267+0000","last_peered":"2023-10-06T11:41:56.520267+0000","last_clean":"2023-10-06T11:41:56.520267+0000","last_became_active":"2023-10-06T11:41:56.516640+0000","last_became_peered":"2023-10-06T11:41:56.516640+0000","last_unstale":"2023-10-06T11:41:56.520267+0000","last_undegraded":"2023-10-06T11:41:56.520267+0000","last_fullsized":"2023-10-06T11:41:56.520267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520067+0000","last_change":"2023-10-06T11:41:56.516139+0000","last_active":"2023-10-06T11:41:56.520067+0000","last_peered":"2023-10-06T11:41:56.520067+0000","last_clean":"2023-10-06T11:41:56.520067+0000","last_became_active":"2023-10-06T11:41:56.514921+0000","last_became_peered":"2023-10-06T11:41:56.514921+0000","last_unstale":"2023-10-06T11:41:56.520067+0000","last_undegraded":"2023-10-06T11:41:56.520067+0000","last_fullsized":"2023-10-06T11:41:56.520067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":671324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4649999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0229999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9600000000000009}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.6189999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6849999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8239999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.9279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:05.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.487+0000 7fca43d36700 1 Processor -- start 2023-10-06T11:42:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca43d36700 1 -- start start 2023-10-06T11:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca43d36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 0x7fca3c0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca43d36700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0ccb20 con 0x7fca3c0cd870 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca43d36700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0cbf10 con 0x7fca3c15cc20 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca43d36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0b3110 con 0x7fca3c0c65a0 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.488+0000 7fca412d1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca3c0cd870 0x7fca3c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57062/0 (socket says 172.21.15.16:57062) 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca412d1700 1 -- 172.21.15.16:0/1011569675 learned_addr learned my addr 172.21.15.16:0/1011569675 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca422d3700 1 --2- 172.21.15.16:0/1011569675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 0x7fca3c0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556034972 0 0) 0x7fca3c0ccb20 con 0x7fca3c0cd870 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca24003570 con 0x7fca3c0cd870 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110267707 0 0) 0x7fca3c0cbf10 con 0x7fca3c15cc20 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3c0ccb20 con 0x7fca3c15cc20 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2936597296 0 0) 0x7fca24003570 con 0x7fca3c0cd870 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca422d3700 1 -- 172.21.15.16:0/1011569675 >> v1:172.21.15.177:6789/0 conn(0x7fca3c15cc20 legacy=0x7fca3c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca422d3700 1 -- 172.21.15.16:0/1011569675 >> v1:172.21.15.16:6789/0 conn(0x7fca3c0cd870 legacy=0x7fca3c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca422d3700 1 -- 172.21.15.16:0/1011569675 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c0cc1f0 con 0x7fca3c0c65a0 2023-10-06T11:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca422d3700 1 --2- 172.21.15.16:0/1011569675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 0x7fca3c0cf450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009e90 tx=0x7fca2c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1661de2b0cfa920a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.489+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00e070 con 0x7fca3c0c65a0 2023-10-06T11:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00b870 con 0x7fca3c0c65a0 2023-10-06T11:42:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca40ad0700 1 -- 172.21.15.16:0/1011569675 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30001470 con 0x7fca3c0c65a0 2023-10-06T11:42:05.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 -- 172.21.15.16:0/1011569675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 msgr2=0x7fca3c0cf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 --2- 172.21.15.16:0/1011569675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 0x7fca3c0cf450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009e90 tx=0x7fca2c009780 comp rx=0 tx=0).stop 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 -- 172.21.15.16:0/1011569675 shutdown_connections 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 --2- 172.21.15.16:0/1011569675 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0c65a0 0x7fca3c0cf450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 -- 172.21.15.16:0/1011569675 >> 172.21.15.16:0/1011569675 conn(0x7fca3c099b40 msgr2=0x7fca3c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 -- 172.21.15.16:0/1011569675 shutdown_connections 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.490+0000 7fca43d36700 1 -- 172.21.15.16:0/1011569675 wait complete. 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.491+0000 7fca43d36700 1 Processor -- start 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.491+0000 7fca43d36700 1 -- start start 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0cd870 0x7fca3c15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca3c15cc20 0x7fca3c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c0c8ca0 con 0x7fca3c0c65a0 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15bc00 con 0x7fca3c15cc20 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca43d36700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c152ab0 con 0x7fca3c0cd870 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43416/0 (socket says 172.21.15.16:43416) 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 -- 172.21.15.16:0/1433376868 learned_addr learned my addr 172.21.15.16:0/1433376868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca422d3700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca3c15cc20 0x7fca3c15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca412d1700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0cd870 0x7fca3c15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 -- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0cd870 msgr2=0x7fca3c15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0cd870 0x7fca3c15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 -- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca3c15cc20 msgr2=0x7fca3c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca3c15cc20 0x7fca3c15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 -- 172.21.15.16:0/1433376868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009a20 con 0x7fca3c0c65a0 2023-10-06T11:42:05.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.492+0000 7fca41ad2700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c790 tx=0x7fca34007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.493+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340011b0 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.493+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34011590 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.493+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c15bea0 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.493+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011950 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.493+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c15c460 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.494+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c04c380 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.494+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca3401b3b0 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.494+0000 7fca39ffb700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 0x7fca28069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.495+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca34083b10 con 0x7fca3c0c65a0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.495+0000 7fca412d1700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 0x7fca28069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.495+0000 7fca412d1700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 0x7fca28069410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca30001ed0 tx=0x7fca300018a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.499+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca34050c20 con 0x7fca3c0c65a0 2023-10-06T11:42:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.688+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca3c0c8de0 con 0x7fca28066f60 2023-10-06T11:42:05.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.697+0000 7fca39ffb700 1 -- 172.21.15.16:0/1433376868 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286837 (secure 0 0 0) 0x7fca300058c0 con 0x7fca28066f60 2023-10-06T11:42:05.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 msgr2=0x7fca28069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 0x7fca28069410 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca30001ed0 tx=0x7fca300018a0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 msgr2=0x7fca3c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c790 tx=0x7fca34007e10 comp rx=0 tx=0).stop 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 shutdown_connections 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3c0c65a0 0x7fca3c15b6c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fca28066f60 0x7fca28069410 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca3c15cc20 0x7fca3c15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 --2- 172.21.15.16:0/1433376868 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca3c0cd870 0x7fca3c15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 >> 172.21.15.16:0/1433376868 conn(0x7fca3c099b40 msgr2=0x7fca3c15ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 shutdown_connections 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.700+0000 7fca43d36700 1 -- 172.21.15.16:0/1433376868 wait complete. 2023-10-06T11:42:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:05.714 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T11:42:04.061745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381212,"kb_used_data":2908,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593508,"statfs":{"total":773094113280,"available":770655752192,"internally_reserved":0,"allocated":2977792,"data_stored":1816046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002058"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517366+0000","last_change":"2023-10-06T11:41:56.510681+0000","last_active":"2023-10-06T11:41:56.517366+0000","last_peered":"2023-10-06T11:41:56.517366+0000","last_clean":"2023-10-06T11:41:56.517366+0000","last_became_active":"2023-10-06T11:41:56.510274+0000","last_became_peered":"2023-10-06T11:41:56.510274+0000","last_unstale":"2023-10-06T11:41:56.517366+0000","last_undegraded":"2023-10-06T11:41:56.517366+0000","last_fullsized":"2023-10-06T11:41:56.517366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:25.325169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519077+0000","last_change":"2023-10-06T11:41:56.512482+0000","last_active":"2023-10-06T11:41:56.519077+0000","last_peered":"2023-10-06T11:41:56.519077+0000","last_clean":"2023-10-06T11:41:56.519077+0000","last_became_active":"2023-10-06T11:41:56.512222+0000","last_became_peered":"2023-10-06T11:41:56.512222+0000","last_unstale":"2023-10-06T11:41:56.519077+0000","last_undegraded":"2023-10-06T11:41:56.519077+0000","last_fullsized":"2023-10-06T11:41:56.519077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513731+0000","last_change":"2023-10-06T11:41:56.509931+0000","last_active":"2023-10-06T11:41:56.513731+0000","last_peered":"2023-10-06T11:41:56.513731+0000","last_clean":"2023-10-06T11:41:56.513731+0000","last_became_active":"2023-10-06T11:41:56.509393+0000","last_became_peered":"2023-10-06T11:41:56.509393+0000","last_unstale":"2023-10-06T11:41:56.513731+0000","last_undegraded":"2023-10-06T11:41:56.513731+0000","last_fullsized":"2023-10-06T11:41:56.513731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:56.452443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516425+0000","last_change":"2023-10-06T11:41:56.511208+0000","last_active":"2023-10-06T11:41:56.516425+0000","last_peered":"2023-10-06T11:41:56.516425+0000","last_clean":"2023-10-06T11:41:56.516425+0000","last_became_active":"2023-10-06T11:41:56.510934+0000","last_became_peered":"2023-10-06T11:41:56.510934+0000","last_unstale":"2023-10-06T11:41:56.516425+0000","last_undegraded":"2023-10-06T11:41:56.516425+0000","last_fullsized":"2023-10-06T11:41:56.516425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:21.656559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519759+0000","last_change":"2023-10-06T11:41:56.516474+0000","last_active":"2023-10-06T11:41:56.519759+0000","last_peered":"2023-10-06T11:41:56.519759+0000","last_clean":"2023-10-06T11:41:56.519759+0000","last_became_active":"2023-10-06T11:41:56.516168+0000","last_became_peered":"2023-10-06T11:41:56.516168+0000","last_unstale":"2023-10-06T11:41:56.519759+0000","last_undegraded":"2023-10-06T11:41:56.519759+0000","last_fullsized":"2023-10-06T11:41:56.519759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513822+0000","last_change":"2023-10-06T11:41:56.509889+0000","last_active":"2023-10-06T11:41:56.513822+0000","last_peered":"2023-10-06T11:41:56.513822+0000","last_clean":"2023-10-06T11:41:56.513822+0000","last_became_active":"2023-10-06T11:41:56.509181+0000","last_became_peered":"2023-10-06T11:41:56.509181+0000","last_unstale":"2023-10-06T11:41:56.513822+0000","last_undegraded":"2023-10-06T11:41:56.513822+0000","last_fullsized":"2023-10-06T11:41:56.513822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:17:42.296263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522246+0000","last_change":"2023-10-06T11:41:56.518850+0000","last_active":"2023-10-06T11:41:56.522246+0000","last_peered":"2023-10-06T11:41:56.522246+0000","last_clean":"2023-10-06T11:41:56.522246+0000","last_became_active":"2023-10-06T11:41:56.518578+0000","last_became_peered":"2023-10-06T11:41:56.518578+0000","last_unstale":"2023-10-06T11:41:56.522246+0000","last_undegraded":"2023-10-06T11:41:56.522246+0000","last_fullsized":"2023-10-06T11:41:56.522246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514003+0000","last_change":"2023-10-06T11:41:56.508470+0000","last_active":"2023-10-06T11:41:56.514003+0000","last_peered":"2023-10-06T11:41:56.514003+0000","last_clean":"2023-10-06T11:41:56.514003+0000","last_became_active":"2023-10-06T11:41:56.508167+0000","last_became_peered":"2023-10-06T11:41:56.508167+0000","last_unstale":"2023-10-06T11:41:56.514003+0000","last_undegraded":"2023-10-06T11:41:56.514003+0000","last_fullsized":"2023-10-06T11:41:56.514003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519630+0000","last_change":"2023-10-06T11:41:56.517434+0000","last_active":"2023-10-06T11:41:56.519630+0000","last_peered":"2023-10-06T11:41:56.519630+0000","last_clean":"2023-10-06T11:41:56.519630+0000","last_became_active":"2023-10-06T11:41:56.517143+0000","last_became_peered":"2023-10-06T11:41:56.517143+0000","last_unstale":"2023-10-06T11:41:56.519630+0000","last_undegraded":"2023-10-06T11:41:56.519630+0000","last_fullsized":"2023-10-06T11:41:56.519630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514049+0000","last_change":"2023-10-06T11:41:56.510953+0000","last_active":"2023-10-06T11:41:56.514049+0000","last_peered":"2023-10-06T11:41:56.514049+0000","last_clean":"2023-10-06T11:41:56.514049+0000","last_became_active":"2023-10-06T11:41:56.510684+0000","last_became_peered":"2023-10-06T11:41:56.510684+0000","last_unstale":"2023-10-06T11:41:56.514049+0000","last_undegraded":"2023-10-06T11:41:56.514049+0000","last_fullsized":"2023-10-06T11:41:56.514049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516780+0000","last_change":"2023-10-06T11:41:56.511114+0000","last_active":"2023-10-06T11:41:56.516780+0000","last_peered":"2023-10-06T11:41:56.516780+0000","last_clean":"2023-10-06T11:41:56.516780+0000","last_became_active":"2023-10-06T11:41:56.510653+0000","last_became_peered":"2023-10-06T11:41:56.510653+0000","last_unstale":"2023-10-06T11:41:56.516780+0000","last_undegraded":"2023-10-06T11:41:56.516780+0000","last_fullsized":"2023-10-06T11:41:56.516780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518118+0000","last_change":"2023-10-06T11:41:56.512980+0000","last_active":"2023-10-06T11:41:56.518118+0000","last_peered":"2023-10-06T11:41:56.518118+0000","last_clean":"2023-10-06T11:41:56.518118+0000","last_became_active":"2023-10-06T11:41:56.512699+0000","last_became_peered":"2023-10-06T11:41:56.512699+0000","last_unstale":"2023-10-06T11:41:56.518118+0000","last_undegraded":"2023-10-06T11:41:56.518118+0000","last_fullsized":"2023-10-06T11:41:56.518118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:20.998002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518013+0000","last_change":"2023-10-06T11:41:56.512383+0000","last_active":"2023-10-06T11:41:56.518013+0000","last_peered":"2023-10-06T11:41:56.518013+0000","last_clean":"2023-10-06T11:41:56.518013+0000","last_became_active":"2023-10-06T11:41:56.512086+0000","last_became_peered":"2023-10-06T11:41:56.512086+0000","last_unstale":"2023-10-06T11:41:56.518013+0000","last_undegraded":"2023-10-06T11:41:56.518013+0000","last_fullsized":"2023-10-06T11:41:56.518013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:44.564348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514263+0000","last_change":"2023-10-06T11:41:56.510312+0000","last_active":"2023-10-06T11:41:56.514263+0000","last_peered":"2023-10-06T11:41:56.514263+0000","last_clean":"2023-10-06T11:41:56.514263+0000","last_became_active":"2023-10-06T11:41:56.510046+0000","last_became_peered":"2023-10-06T11:41:56.510046+0000","last_unstale":"2023-10-06T11:41:56.514263+0000","last_undegraded":"2023-10-06T11:41:56.514263+0000","last_fullsized":"2023-10-06T11:41:56.514263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.522268+0000","last_change":"2023-10-06T11:41:56.518929+0000","last_active":"2023-10-06T11:41:56.522268+0000","last_peered":"2023-10-06T11:41:56.522268+0000","last_clean":"2023-10-06T11:41:56.522268+0000","last_became_active":"2023-10-06T11:41:56.518671+0000","last_became_peered":"2023-10-06T11:41:56.518671+0000","last_unstale":"2023-10-06T11:41:56.522268+0000","last_undegraded":"2023-10-06T11:41:56.522268+0000","last_fullsized":"2023-10-06T11:41:56.522268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513758+0000","last_change":"2023-10-06T11:41:56.509452+0000","last_active":"2023-10-06T11:41:56.513758+0000","last_peered":"2023-10-06T11:41:56.513758+0000","last_clean":"2023-10-06T11:41:56.513758+0000","last_became_active":"2023-10-06T11:41:56.509141+0000","last_became_peered":"2023-10-06T11:41:56.509141+0000","last_unstale":"2023-10-06T11:41:56.513758+0000","last_undegraded":"2023-10-06T11:41:56.513758+0000","last_fullsized":"2023-10-06T11:41:56.513758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515869+0000","last_change":"2023-10-06T11:41:56.511167+0000","last_active":"2023-10-06T11:41:56.515869+0000","last_peered":"2023-10-06T11:41:56.515869+0000","last_clean":"2023-10-06T11:41:56.515869+0000","last_became_active":"2023-10-06T11:41:56.510849+0000","last_became_peered":"2023-10-06T11:41:56.510849+0000","last_unstale":"2023-10-06T11:41:56.515869+0000","last_undegraded":"2023-10-06T11:41:56.515869+0000","last_fullsized":"2023-10-06T11:41:56.515869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516793+0000","last_change":"2023-10-06T11:41:56.511248+0000","last_active":"2023-10-06T11:41:56.516793+0000","last_peered":"2023-10-06T11:41:56.516793+0000","last_clean":"2023-10-06T11:41:56.516793+0000","last_became_active":"2023-10-06T11:41:56.510023+0000","last_became_peered":"2023-10-06T11:41:56.510023+0000","last_unstale":"2023-10-06T11:41:56.516793+0000","last_undegraded":"2023-10-06T11:41:56.516793+0000","last_fullsized":"2023-10-06T11:41:56.516793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519577+0000","last_change":"2023-10-06T11:41:56.513818+0000","last_active":"2023-10-06T11:41:56.519577+0000","last_peered":"2023-10-06T11:41:56.519577+0000","last_clean":"2023-10-06T11:41:56.519577+0000","last_became_active":"2023-10-06T11:41:56.513508+0000","last_became_peered":"2023-10-06T11:41:56.513508+0000","last_unstale":"2023-10-06T11:41:56.519577+0000","last_undegraded":"2023-10-06T11:41:56.519577+0000","last_fullsized":"2023-10-06T11:41:56.519577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:25.568072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519973+0000","last_change":"2023-10-06T11:41:56.514028+0000","last_active":"2023-10-06T11:41:56.519973+0000","last_peered":"2023-10-06T11:41:56.519973+0000","last_clean":"2023-10-06T11:41:56.519973+0000","last_became_active":"2023-10-06T11:41:56.513176+0000","last_became_peered":"2023-10-06T11:41:56.513176+0000","last_unstale":"2023-10-06T11:41:56.519973+0000","last_undegraded":"2023-10-06T11:41:56.519973+0000","last_fullsized":"2023-10-06T11:41:56.519973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516670+0000","last_change":"2023-10-06T11:41:56.513150+0000","last_active":"2023-10-06T11:41:56.516670+0000","last_peered":"2023-10-06T11:41:56.516670+0000","last_clean":"2023-10-06T11:41:56.516670+0000","last_became_active":"2023-10-06T11:41:56.512844+0000","last_became_peered":"2023-10-06T11:41:56.512844+0000","last_unstale":"2023-10-06T11:41:56.516670+0000","last_undegraded":"2023-10-06T11:41:56.516670+0000","last_fullsized":"2023-10-06T11:41:56.516670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519663+0000","last_change":"2023-10-06T11:41:56.512179+0000","last_active":"2023-10-06T11:41:56.519663+0000","last_peered":"2023-10-06T11:41:56.519663+0000","last_clean":"2023-10-06T11:41:56.519663+0000","last_became_active":"2023-10-06T11:41:56.511885+0000","last_became_peered":"2023-10-06T11:41:56.511885+0000","last_unstale":"2023-10-06T11:41:56.519663+0000","last_undegraded":"2023-10-06T11:41:56.519663+0000","last_fullsized":"2023-10-06T11:41:56.519663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520009+0000","last_change":"2023-10-06T11:41:56.514447+0000","last_active":"2023-10-06T11:41:56.520009+0000","last_peered":"2023-10-06T11:41:56.520009+0000","last_clean":"2023-10-06T11:41:56.520009+0000","last_became_active":"2023-10-06T11:41:56.514134+0000","last_became_peered":"2023-10-06T11:41:56.514134+0000","last_unstale":"2023-10-06T11:41:56.520009+0000","last_undegraded":"2023-10-06T11:41:56.520009+0000","last_fullsized":"2023-10-06T11:41:56.520009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516608+0000","last_change":"2023-10-06T11:41:56.513062+0000","last_active":"2023-10-06T11:41:56.516608+0000","last_peered":"2023-10-06T11:41:56.516608+0000","last_clean":"2023-10-06T11:41:56.516608+0000","last_became_active":"2023-10-06T11:41:56.512812+0000","last_became_peered":"2023-10-06T11:41:56.512812+0000","last_unstale":"2023-10-06T11:41:56.516608+0000","last_undegraded":"2023-10-06T11:41:56.516608+0000","last_fullsized":"2023-10-06T11:41:56.516608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517749+0000","last_change":"2023-10-06T11:41:56.514788+0000","last_active":"2023-10-06T11:41:56.517749+0000","last_peered":"2023-10-06T11:41:56.517749+0000","last_clean":"2023-10-06T11:41:56.517749+0000","last_became_active":"2023-10-06T11:41:56.514494+0000","last_became_peered":"2023-10-06T11:41:56.514494+0000","last_unstale":"2023-10-06T11:41:56.517749+0000","last_undegraded":"2023-10-06T11:41:56.517749+0000","last_fullsized":"2023-10-06T11:41:56.517749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:42.124093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516257+0000","last_change":"2023-10-06T11:41:56.511137+0000","last_active":"2023-10-06T11:41:56.516257+0000","last_peered":"2023-10-06T11:41:56.516257+0000","last_clean":"2023-10-06T11:41:56.516257+0000","last_became_active":"2023-10-06T11:41:56.510826+0000","last_became_peered":"2023-10-06T11:41:56.510826+0000","last_unstale":"2023-10-06T11:41:56.516257+0000","last_undegraded":"2023-10-06T11:41:56.516257+0000","last_fullsized":"2023-10-06T11:41:56.516257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519152+0000","last_change":"2023-10-06T11:41:56.513345+0000","last_active":"2023-10-06T11:41:56.519152+0000","last_peered":"2023-10-06T11:41:56.519152+0000","last_clean":"2023-10-06T11:41:56.519152+0000","last_became_active":"2023-10-06T11:41:56.513068+0000","last_became_peered":"2023-10-06T11:41:56.513068+0000","last_unstale":"2023-10-06T11:41:56.519152+0000","last_undegraded":"2023-10-06T11:41:56.519152+0000","last_fullsized":"2023-10-06T11:41:56.519152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.529014+0000","last_change":"2023-10-06T11:41:55.504888+0000","last_active":"2023-10-06T11:41:56.529014+0000","last_peered":"2023-10-06T11:41:56.529014+0000","last_clean":"2023-10-06T11:41:56.529014+0000","last_became_active":"2023-10-06T11:41:55.504602+0000","last_became_peered":"2023-10-06T11:41:55.504602+0000","last_unstale":"2023-10-06T11:41:56.529014+0000","last_undegraded":"2023-10-06T11:41:56.529014+0000","last_fullsized":"2023-10-06T11:41:56.529014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.518587+0000","last_change":"2023-10-06T11:41:56.512500+0000","last_active":"2023-10-06T11:41:56.518587+0000","last_peered":"2023-10-06T11:41:56.518587+0000","last_clean":"2023-10-06T11:41:56.518587+0000","last_became_active":"2023-10-06T11:41:56.512213+0000","last_became_peered":"2023-10-06T11:41:56.512213+0000","last_unstale":"2023-10-06T11:41:56.518587+0000","last_undegraded":"2023-10-06T11:41:56.518587+0000","last_fullsized":"2023-10-06T11:41:56.518587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497430+0000","last_change":"2023-10-06T11:41:55.505880+0000","last_active":"2023-10-06T11:41:56.497430+0000","last_peered":"2023-10-06T11:41:56.497430+0000","last_clean":"2023-10-06T11:41:56.497430+0000","last_became_active":"2023-10-06T11:41:55.504907+0000","last_became_peered":"2023-10-06T11:41:55.504907+0000","last_unstale":"2023-10-06T11:41:56.497430+0000","last_undegraded":"2023-10-06T11:41:56.497430+0000","last_fullsized":"2023-10-06T11:41:56.497430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519692+0000","last_change":"2023-10-06T11:41:56.514094+0000","last_active":"2023-10-06T11:41:56.519692+0000","last_peered":"2023-10-06T11:41:56.519692+0000","last_clean":"2023-10-06T11:41:56.519692+0000","last_became_active":"2023-10-06T11:41:56.513833+0000","last_became_peered":"2023-10-06T11:41:56.513833+0000","last_unstale":"2023-10-06T11:41:56.519692+0000","last_undegraded":"2023-10-06T11:41:56.519692+0000","last_fullsized":"2023-10-06T11:41:56.519692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503895+0000","last_change":"2023-10-06T11:41:55.504864+0000","last_active":"2023-10-06T11:41:56.503895+0000","last_peered":"2023-10-06T11:41:56.503895+0000","last_clean":"2023-10-06T11:41:56.503895+0000","last_became_active":"2023-10-06T11:41:55.504166+0000","last_became_peered":"2023-10-06T11:41:55.504166+0000","last_unstale":"2023-10-06T11:41:56.503895+0000","last_undegraded":"2023-10-06T11:41:56.503895+0000","last_fullsized":"2023-10-06T11:41:56.503895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514214+0000","last_change":"2023-10-06T11:41:56.509697+0000","last_active":"2023-10-06T11:41:56.514214+0000","last_peered":"2023-10-06T11:41:56.514214+0000","last_clean":"2023-10-06T11:41:56.514214+0000","last_became_active":"2023-10-06T11:41:56.509414+0000","last_became_peered":"2023-10-06T11:41:56.509414+0000","last_unstale":"2023-10-06T11:41:56.514214+0000","last_undegraded":"2023-10-06T11:41:56.514214+0000","last_fullsized":"2023-10-06T11:41:56.514214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507789+0000","last_change":"2023-10-06T11:41:55.496678+0000","last_active":"2023-10-06T11:41:56.507789+0000","last_peered":"2023-10-06T11:41:56.507789+0000","last_clean":"2023-10-06T11:41:56.507789+0000","last_became_active":"2023-10-06T11:41:55.494896+0000","last_became_peered":"2023-10-06T11:41:55.494896+0000","last_unstale":"2023-10-06T11:41:56.507789+0000","last_undegraded":"2023-10-06T11:41:56.507789+0000","last_fullsized":"2023-10-06T11:41:56.507789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.517092+0000","last_change":"2023-10-06T11:41:56.512202+0000","last_active":"2023-10-06T11:41:56.517092+0000","last_peered":"2023-10-06T11:41:56.517092+0000","last_clean":"2023-10-06T11:41:56.517092+0000","last_became_active":"2023-10-06T11:41:56.511913+0000","last_became_peered":"2023-10-06T11:41:56.511913+0000","last_unstale":"2023-10-06T11:41:56.517092+0000","last_undegraded":"2023-10-06T11:41:56.517092+0000","last_fullsized":"2023-10-06T11:41:56.517092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:07:03.191182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502121+0000","last_change":"2023-10-06T11:41:55.500940+0000","last_active":"2023-10-06T11:41:56.502121+0000","last_peered":"2023-10-06T11:41:56.502121+0000","last_clean":"2023-10-06T11:41:56.502121+0000","last_became_active":"2023-10-06T11:41:55.500652+0000","last_became_peered":"2023-10-06T11:41:55.500652+0000","last_unstale":"2023-10-06T11:41:56.502121+0000","last_undegraded":"2023-10-06T11:41:56.502121+0000","last_fullsized":"2023-10-06T11:41:56.502121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520515+0000","last_change":"2023-10-06T11:41:56.515837+0000","last_active":"2023-10-06T11:41:56.520515+0000","last_peered":"2023-10-06T11:41:56.520515+0000","last_clean":"2023-10-06T11:41:56.520515+0000","last_became_active":"2023-10-06T11:41:56.515511+0000","last_became_peered":"2023-10-06T11:41:56.515511+0000","last_unstale":"2023-10-06T11:41:56.520515+0000","last_undegraded":"2023-10-06T11:41:56.520515+0000","last_fullsized":"2023-10-06T11:41:56.520515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534860+0000","last_change":"2023-10-06T11:41:55.509167+0000","last_active":"2023-10-06T11:41:56.534860+0000","last_peered":"2023-10-06T11:41:56.534860+0000","last_clean":"2023-10-06T11:41:56.534860+0000","last_became_active":"2023-10-06T11:41:55.508968+0000","last_became_peered":"2023-10-06T11:41:55.508968+0000","last_unstale":"2023-10-06T11:41:56.534860+0000","last_undegraded":"2023-10-06T11:41:56.534860+0000","last_fullsized":"2023-10-06T11:41:56.534860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:13.549315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520107+0000","last_change":"2023-10-06T11:41:56.515835+0000","last_active":"2023-10-06T11:41:56.520107+0000","last_peered":"2023-10-06T11:41:56.520107+0000","last_clean":"2023-10-06T11:41:56.520107+0000","last_became_active":"2023-10-06T11:41:56.514755+0000","last_became_peered":"2023-10-06T11:41:56.514755+0000","last_unstale":"2023-10-06T11:41:56.520107+0000","last_undegraded":"2023-10-06T11:41:56.520107+0000","last_fullsized":"2023-10-06T11:41:56.520107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.536294+0000","last_change":"2023-10-06T11:41:55.503423+0000","last_active":"2023-10-06T11:41:56.536294+0000","last_peered":"2023-10-06T11:41:56.536294+0000","last_clean":"2023-10-06T11:41:56.536294+0000","last_became_active":"2023-10-06T11:41:55.499943+0000","last_became_peered":"2023-10-06T11:41:55.499943+0000","last_unstale":"2023-10-06T11:41:56.536294+0000","last_undegraded":"2023-10-06T11:41:56.536294+0000","last_fullsized":"2023-10-06T11:41:56.536294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519939+0000","last_change":"2023-10-06T11:41:56.513385+0000","last_active":"2023-10-06T11:41:56.519939+0000","last_peered":"2023-10-06T11:41:56.519939+0000","last_clean":"2023-10-06T11:41:56.519939+0000","last_became_active":"2023-10-06T11:41:56.513091+0000","last_became_peered":"2023-10-06T11:41:56.513091+0000","last_unstale":"2023-10-06T11:41:56.519939+0000","last_undegraded":"2023-10-06T11:41:56.519939+0000","last_fullsized":"2023-10-06T11:41:56.519939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:12:31.044745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534779+0000","last_change":"2023-10-06T11:41:55.508640+0000","last_active":"2023-10-06T11:41:56.534779+0000","last_peered":"2023-10-06T11:41:56.534779+0000","last_clean":"2023-10-06T11:41:56.534779+0000","last_became_active":"2023-10-06T11:41:55.508359+0000","last_became_peered":"2023-10-06T11:41:55.508359+0000","last_unstale":"2023-10-06T11:41:56.534779+0000","last_undegraded":"2023-10-06T11:41:56.534779+0000","last_fullsized":"2023-10-06T11:41:56.534779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:02:20.652268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520221+0000","last_change":"2023-10-06T11:41:56.518196+0000","last_active":"2023-10-06T11:41:56.520221+0000","last_peered":"2023-10-06T11:41:56.520221+0000","last_clean":"2023-10-06T11:41:56.520221+0000","last_became_active":"2023-10-06T11:41:56.517922+0000","last_became_peered":"2023-10-06T11:41:56.517922+0000","last_unstale":"2023-10-06T11:41:56.520221+0000","last_undegraded":"2023-10-06T11:41:56.520221+0000","last_fullsized":"2023-10-06T11:41:56.520221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507442+0000","last_change":"2023-10-06T11:41:55.498881+0000","last_active":"2023-10-06T11:41:56.507442+0000","last_peered":"2023-10-06T11:41:56.507442+0000","last_clean":"2023-10-06T11:41:56.507442+0000","last_became_active":"2023-10-06T11:41:55.498646+0000","last_became_peered":"2023-10-06T11:41:55.498646+0000","last_unstale":"2023-10-06T11:41:56.507442+0000","last_undegraded":"2023-10-06T11:41:56.507442+0000","last_fullsized":"2023-10-06T11:41:56.507442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.521939+0000","last_change":"2023-10-06T11:41:56.514291+0000","last_active":"2023-10-06T11:41:56.521939+0000","last_peered":"2023-10-06T11:41:56.521939+0000","last_clean":"2023-10-06T11:41:56.521939+0000","last_became_active":"2023-10-06T11:41:56.513811+0000","last_became_peered":"2023-10-06T11:41:56.513811+0000","last_unstale":"2023-10-06T11:41:56.521939+0000","last_undegraded":"2023-10-06T11:41:56.521939+0000","last_fullsized":"2023-10-06T11:41:56.521939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:26:28.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506360+0000","last_change":"2023-10-06T11:41:55.510456+0000","last_active":"2023-10-06T11:41:56.506360+0000","last_peered":"2023-10-06T11:41:56.506360+0000","last_clean":"2023-10-06T11:41:56.506360+0000","last_became_active":"2023-10-06T11:41:55.510148+0000","last_became_peered":"2023-10-06T11:41:55.510148+0000","last_unstale":"2023-10-06T11:41:56.506360+0000","last_undegraded":"2023-10-06T11:41:56.506360+0000","last_fullsized":"2023-10-06T11:41:56.506360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519708+0000","last_change":"2023-10-06T11:41:56.516787+0000","last_active":"2023-10-06T11:41:56.519708+0000","last_peered":"2023-10-06T11:41:56.519708+0000","last_clean":"2023-10-06T11:41:56.519708+0000","last_became_active":"2023-10-06T11:41:56.516510+0000","last_became_peered":"2023-10-06T11:41:56.516510+0000","last_unstale":"2023-10-06T11:41:56.519708+0000","last_undegraded":"2023-10-06T11:41:56.519708+0000","last_fullsized":"2023-10-06T11:41:56.519708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:44:50.674654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.532331+0000","last_change":"2023-10-06T11:41:55.502794+0000","last_active":"2023-10-06T11:41:56.532331+0000","last_peered":"2023-10-06T11:41:56.532331+0000","last_clean":"2023-10-06T11:41:56.532331+0000","last_became_active":"2023-10-06T11:41:55.502522+0000","last_became_peered":"2023-10-06T11:41:55.502522+0000","last_unstale":"2023-10-06T11:41:56.532331+0000","last_undegraded":"2023-10-06T11:41:56.532331+0000","last_fullsized":"2023-10-06T11:41:56.532331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520360+0000","last_change":"2023-10-06T11:41:56.515679+0000","last_active":"2023-10-06T11:41:56.520360+0000","last_peered":"2023-10-06T11:41:56.520360+0000","last_clean":"2023-10-06T11:41:56.520360+0000","last_became_active":"2023-10-06T11:41:56.515167+0000","last_became_peered":"2023-10-06T11:41:56.515167+0000","last_unstale":"2023-10-06T11:41:56.520360+0000","last_undegraded":"2023-10-06T11:41:56.520360+0000","last_fullsized":"2023-10-06T11:41:56.520360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:16:07.986581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534434+0000","last_change":"2023-10-06T11:41:55.506331+0000","last_active":"2023-10-06T11:41:56.534434+0000","last_peered":"2023-10-06T11:41:56.534434+0000","last_clean":"2023-10-06T11:41:56.534434+0000","last_became_active":"2023-10-06T11:41:55.506044+0000","last_became_peered":"2023-10-06T11:41:55.506044+0000","last_unstale":"2023-10-06T11:41:56.534434+0000","last_undegraded":"2023-10-06T11:41:56.534434+0000","last_fullsized":"2023-10-06T11:41:56.534434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514023+0000","last_change":"2023-10-06T11:41:56.509291+0000","last_active":"2023-10-06T11:41:56.514023+0000","last_peered":"2023-10-06T11:41:56.514023+0000","last_clean":"2023-10-06T11:41:56.514023+0000","last_became_active":"2023-10-06T11:41:56.509004+0000","last_became_peered":"2023-10-06T11:41:56.509004+0000","last_unstale":"2023-10-06T11:41:56.514023+0000","last_undegraded":"2023-10-06T11:41:56.514023+0000","last_fullsized":"2023-10-06T11:41:56.514023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534956+0000","last_change":"2023-10-06T11:41:55.510357+0000","last_active":"2023-10-06T11:41:56.534956+0000","last_peered":"2023-10-06T11:41:56.534956+0000","last_clean":"2023-10-06T11:41:56.534956+0000","last_became_active":"2023-10-06T11:41:55.510053+0000","last_became_peered":"2023-10-06T11:41:55.510053+0000","last_unstale":"2023-10-06T11:41:56.534956+0000","last_undegraded":"2023-10-06T11:41:56.534956+0000","last_fullsized":"2023-10-06T11:41:56.534956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516748+0000","last_change":"2023-10-06T11:41:56.510920+0000","last_active":"2023-10-06T11:41:56.516748+0000","last_peered":"2023-10-06T11:41:56.516748+0000","last_clean":"2023-10-06T11:41:56.516748+0000","last_became_active":"2023-10-06T11:41:56.510620+0000","last_became_peered":"2023-10-06T11:41:56.510620+0000","last_unstale":"2023-10-06T11:41:56.516748+0000","last_undegraded":"2023-10-06T11:41:56.516748+0000","last_fullsized":"2023-10-06T11:41:56.516748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:01.202344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534773+0000","last_change":"2023-10-06T11:41:55.501172+0000","last_active":"2023-10-06T11:41:56.534773+0000","last_peered":"2023-10-06T11:41:56.534773+0000","last_clean":"2023-10-06T11:41:56.534773+0000","last_became_active":"2023-10-06T11:41:55.500891+0000","last_became_peered":"2023-10-06T11:41:55.500891+0000","last_unstale":"2023-10-06T11:41:56.534773+0000","last_undegraded":"2023-10-06T11:41:56.534773+0000","last_fullsized":"2023-10-06T11:41:56.534773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:13:31.984593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516634+0000","last_change":"2023-10-06T11:41:56.512054+0000","last_active":"2023-10-06T11:41:56.516634+0000","last_peered":"2023-10-06T11:41:56.516634+0000","last_clean":"2023-10-06T11:41:56.516634+0000","last_became_active":"2023-10-06T11:41:56.511385+0000","last_became_peered":"2023-10-06T11:41:56.511385+0000","last_unstale":"2023-10-06T11:41:56.516634+0000","last_undegraded":"2023-10-06T11:41:56.516634+0000","last_fullsized":"2023-10-06T11:41:56.516634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.394264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.501453+0000","last_change":"2023-10-06T11:41:55.505446+0000","last_active":"2023-10-06T11:41:56.501453+0000","last_peered":"2023-10-06T11:41:56.501453+0000","last_clean":"2023-10-06T11:41:56.501453+0000","last_became_active":"2023-10-06T11:41:55.505189+0000","last_became_peered":"2023-10-06T11:41:55.505189+0000","last_unstale":"2023-10-06T11:41:56.501453+0000","last_undegraded":"2023-10-06T11:41:56.501453+0000","last_fullsized":"2023-10-06T11:41:56.501453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519776+0000","last_change":"2023-10-06T11:41:56.517521+0000","last_active":"2023-10-06T11:41:56.519776+0000","last_peered":"2023-10-06T11:41:56.519776+0000","last_clean":"2023-10-06T11:41:56.519776+0000","last_became_active":"2023-10-06T11:41:56.517084+0000","last_became_peered":"2023-10-06T11:41:56.517084+0000","last_unstale":"2023-10-06T11:41:56.519776+0000","last_undegraded":"2023-10-06T11:41:56.519776+0000","last_fullsized":"2023-10-06T11:41:56.519776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531354+0000","last_change":"2023-10-06T11:41:55.509060+0000","last_active":"2023-10-06T11:41:56.531354+0000","last_peered":"2023-10-06T11:41:56.531354+0000","last_clean":"2023-10-06T11:41:56.531354+0000","last_became_active":"2023-10-06T11:41:55.508766+0000","last_became_peered":"2023-10-06T11:41:55.508766+0000","last_unstale":"2023-10-06T11:41:56.531354+0000","last_undegraded":"2023-10-06T11:41:56.531354+0000","last_fullsized":"2023-10-06T11:41:56.531354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:39.945335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.514111+0000","last_change":"2023-10-06T11:41:56.510155+0000","last_active":"2023-10-06T11:41:56.514111+0000","last_peered":"2023-10-06T11:41:56.514111+0000","last_clean":"2023-10-06T11:41:56.514111+0000","last_became_active":"2023-10-06T11:41:56.509859+0000","last_became_peered":"2023-10-06T11:41:56.509859+0000","last_unstale":"2023-10-06T11:41:56.514111+0000","last_undegraded":"2023-10-06T11:41:56.514111+0000","last_fullsized":"2023-10-06T11:41:56.514111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:00:07.370883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507800+0000","last_change":"2023-10-06T11:41:55.511135+0000","last_active":"2023-10-06T11:41:56.507800+0000","last_peered":"2023-10-06T11:41:56.507800+0000","last_clean":"2023-10-06T11:41:56.507800+0000","last_became_active":"2023-10-06T11:41:55.510802+0000","last_became_peered":"2023-10-06T11:41:55.510802+0000","last_unstale":"2023-10-06T11:41:56.507800+0000","last_undegraded":"2023-10-06T11:41:56.507800+0000","last_fullsized":"2023-10-06T11:41:56.507800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516764+0000","last_change":"2023-10-06T11:41:56.510930+0000","last_active":"2023-10-06T11:41:56.516764+0000","last_peered":"2023-10-06T11:41:56.516764+0000","last_clean":"2023-10-06T11:41:56.516764+0000","last_became_active":"2023-10-06T11:41:56.510315+0000","last_became_peered":"2023-10-06T11:41:56.510315+0000","last_unstale":"2023-10-06T11:41:56.516764+0000","last_undegraded":"2023-10-06T11:41:56.516764+0000","last_fullsized":"2023-10-06T11:41:56.516764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:30:29.753304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500145+0000","last_change":"2023-10-06T11:41:55.506086+0000","last_active":"2023-10-06T11:41:56.500145+0000","last_peered":"2023-10-06T11:41:56.500145+0000","last_clean":"2023-10-06T11:41:56.500145+0000","last_became_active":"2023-10-06T11:41:55.505822+0000","last_became_peered":"2023-10-06T11:41:55.505822+0000","last_unstale":"2023-10-06T11:41:56.500145+0000","last_undegraded":"2023-10-06T11:41:56.500145+0000","last_fullsized":"2023-10-06T11:41:56.500145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520047+0000","last_change":"2023-10-06T11:41:56.514492+0000","last_active":"2023-10-06T11:41:56.520047+0000","last_peered":"2023-10-06T11:41:56.520047+0000","last_clean":"2023-10-06T11:41:56.520047+0000","last_became_active":"2023-10-06T11:41:56.513778+0000","last_became_peered":"2023-10-06T11:41:56.513778+0000","last_unstale":"2023-10-06T11:41:56.520047+0000","last_undegraded":"2023-10-06T11:41:56.520047+0000","last_fullsized":"2023-10-06T11:41:56.520047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:56:56.079922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503812+0000","last_change":"2023-10-06T11:41:55.505373+0000","last_active":"2023-10-06T11:41:56.503812+0000","last_peered":"2023-10-06T11:41:56.503812+0000","last_clean":"2023-10-06T11:41:56.503812+0000","last_became_active":"2023-10-06T11:41:55.505092+0000","last_became_peered":"2023-10-06T11:41:55.505092+0000","last_unstale":"2023-10-06T11:41:56.503812+0000","last_undegraded":"2023-10-06T11:41:56.503812+0000","last_fullsized":"2023-10-06T11:41:56.503812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:54.490327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516491+0000","last_change":"2023-10-06T11:41:56.512965+0000","last_active":"2023-10-06T11:41:56.516491+0000","last_peered":"2023-10-06T11:41:56.516491+0000","last_clean":"2023-10-06T11:41:56.516491+0000","last_became_active":"2023-10-06T11:41:56.512684+0000","last_became_peered":"2023-10-06T11:41:56.512684+0000","last_unstale":"2023-10-06T11:41:56.516491+0000","last_undegraded":"2023-10-06T11:41:56.516491+0000","last_fullsized":"2023-10-06T11:41:56.516491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:39.533100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534650+0000","last_change":"2023-10-06T11:41:55.510312+0000","last_active":"2023-10-06T11:41:56.534650+0000","last_peered":"2023-10-06T11:41:56.534650+0000","last_clean":"2023-10-06T11:41:56.534650+0000","last_became_active":"2023-10-06T11:41:55.510044+0000","last_became_peered":"2023-10-06T11:41:55.510044+0000","last_unstale":"2023-10-06T11:41:56.534650+0000","last_undegraded":"2023-10-06T11:41:56.534650+0000","last_fullsized":"2023-10-06T11:41:56.534650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500088+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:41:56.500088+0000","last_peered":"2023-10-06T11:41:56.500088+0000","last_clean":"2023-10-06T11:41:56.500088+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:41:56.500088+0000","last_undegraded":"2023-10-06T11:41:56.500088+0000","last_fullsized":"2023-10-06T11:41:56.500088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519625+0000","last_change":"2023-10-06T11:41:56.515565+0000","last_active":"2023-10-06T11:41:56.519625+0000","last_peered":"2023-10-06T11:41:56.519625+0000","last_clean":"2023-10-06T11:41:56.519625+0000","last_became_active":"2023-10-06T11:41:56.515007+0000","last_became_peered":"2023-10-06T11:41:56.515007+0000","last_unstale":"2023-10-06T11:41:56.519625+0000","last_undegraded":"2023-10-06T11:41:56.519625+0000","last_fullsized":"2023-10-06T11:41:56.519625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:16:59.810980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503231+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:41:56.503231+0000","last_peered":"2023-10-06T11:41:56.503231+0000","last_clean":"2023-10-06T11:41:56.503231+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:41:56.503231+0000","last_undegraded":"2023-10-06T11:41:56.503231+0000","last_fullsized":"2023-10-06T11:41:56.503231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T11:41:58.260002+0000","last_change":"2023-10-06T11:41:55.500074+0000","last_active":"2023-10-06T11:41:58.260002+0000","last_peered":"2023-10-06T11:41:58.260002+0000","last_clean":"2023-10-06T11:41:58.260002+0000","last_became_active":"2023-10-06T11:41:55.499784+0000","last_became_peered":"2023-10-06T11:41:55.499784+0000","last_unstale":"2023-10-06T11:41:58.260002+0000","last_undegraded":"2023-10-06T11:41:58.260002+0000","last_fullsized":"2023-10-06T11:41:58.260002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513641+0000","last_change":"2023-10-06T11:41:56.509030+0000","last_active":"2023-10-06T11:41:56.513641+0000","last_peered":"2023-10-06T11:41:56.513641+0000","last_clean":"2023-10-06T11:41:56.513641+0000","last_became_active":"2023-10-06T11:41:56.508720+0000","last_became_peered":"2023-10-06T11:41:56.508720+0000","last_unstale":"2023-10-06T11:41:56.513641+0000","last_undegraded":"2023-10-06T11:41:56.513641+0000","last_fullsized":"2023-10-06T11:41:56.513641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:51.155604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497934+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:41:56.497934+0000","last_peered":"2023-10-06T11:41:56.497934+0000","last_clean":"2023-10-06T11:41:56.497934+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:41:56.497934+0000","last_undegraded":"2023-10-06T11:41:56.497934+0000","last_fullsized":"2023-10-06T11:41:56.497934+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.506182+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:41:56.506182+0000","last_peered":"2023-10-06T11:41:56.506182+0000","last_clean":"2023-10-06T11:41:56.506182+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:41:56.506182+0000","last_undegraded":"2023-10-06T11:41:56.506182+0000","last_fullsized":"2023-10-06T11:41:56.506182+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507640+0000","last_change":"2023-10-06T11:41:55.506608+0000","last_active":"2023-10-06T11:41:56.507640+0000","last_peered":"2023-10-06T11:41:56.507640+0000","last_clean":"2023-10-06T11:41:56.507640+0000","last_became_active":"2023-10-06T11:41:55.506234+0000","last_became_peered":"2023-10-06T11:41:55.506234+0000","last_unstale":"2023-10-06T11:41:56.507640+0000","last_undegraded":"2023-10-06T11:41:56.507640+0000","last_fullsized":"2023-10-06T11:41:56.507640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519724+0000","last_change":"2023-10-06T11:41:56.516915+0000","last_active":"2023-10-06T11:41:56.519724+0000","last_peered":"2023-10-06T11:41:56.519724+0000","last_clean":"2023-10-06T11:41:56.519724+0000","last_became_active":"2023-10-06T11:41:56.516184+0000","last_became_peered":"2023-10-06T11:41:56.516184+0000","last_unstale":"2023-10-06T11:41:56.519724+0000","last_undegraded":"2023-10-06T11:41:56.519724+0000","last_fullsized":"2023-10-06T11:41:56.519724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:14.987136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505779+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:41:56.505779+0000","last_peered":"2023-10-06T11:41:56.505779+0000","last_clean":"2023-10-06T11:41:56.505779+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:41:56.505779+0000","last_undegraded":"2023-10-06T11:41:56.505779+0000","last_fullsized":"2023-10-06T11:41:56.505779+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497472+0000","last_change":"2023-10-06T11:41:55.506060+0000","last_active":"2023-10-06T11:41:56.497472+0000","last_peered":"2023-10-06T11:41:56.497472+0000","last_clean":"2023-10-06T11:41:56.497472+0000","last_became_active":"2023-10-06T11:41:55.505551+0000","last_became_peered":"2023-10-06T11:41:55.505551+0000","last_unstale":"2023-10-06T11:41:56.497472+0000","last_undegraded":"2023-10-06T11:41:56.497472+0000","last_fullsized":"2023-10-06T11:41:56.497472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:23.340598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.515790+0000","last_change":"2023-10-06T11:41:56.509141+0000","last_active":"2023-10-06T11:41:56.515790+0000","last_peered":"2023-10-06T11:41:56.515790+0000","last_clean":"2023-10-06T11:41:56.515790+0000","last_became_active":"2023-10-06T11:41:56.508814+0000","last_became_peered":"2023-10-06T11:41:56.508814+0000","last_unstale":"2023-10-06T11:41:56.515790+0000","last_undegraded":"2023-10-06T11:41:56.515790+0000","last_fullsized":"2023-10-06T11:41:56.515790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.505943+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:41:56.505943+0000","last_peered":"2023-10-06T11:41:56.505943+0000","last_clean":"2023-10-06T11:41:56.505943+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:41:56.505943+0000","last_undegraded":"2023-10-06T11:41:56.505943+0000","last_fullsized":"2023-10-06T11:41:56.505943+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502767+0000","last_change":"2023-10-06T11:41:55.504016+0000","last_active":"2023-10-06T11:41:56.502767+0000","last_peered":"2023-10-06T11:41:56.502767+0000","last_clean":"2023-10-06T11:41:56.502767+0000","last_became_active":"2023-10-06T11:41:55.503831+0000","last_became_peered":"2023-10-06T11:41:55.503831+0000","last_unstale":"2023-10-06T11:41:56.502767+0000","last_undegraded":"2023-10-06T11:41:56.502767+0000","last_fullsized":"2023-10-06T11:41:56.502767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:28:24.320878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519487+0000","last_change":"2023-10-06T11:41:56.515759+0000","last_active":"2023-10-06T11:41:56.519487+0000","last_peered":"2023-10-06T11:41:56.519487+0000","last_clean":"2023-10-06T11:41:56.519487+0000","last_became_active":"2023-10-06T11:41:56.514780+0000","last_became_peered":"2023-10-06T11:41:56.514780+0000","last_unstale":"2023-10-06T11:41:56.519487+0000","last_undegraded":"2023-10-06T11:41:56.519487+0000","last_fullsized":"2023-10-06T11:41:56.519487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.502819+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:41:56.502819+0000","last_peered":"2023-10-06T11:41:56.502819+0000","last_clean":"2023-10-06T11:41:56.502819+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:41:56.502819+0000","last_undegraded":"2023-10-06T11:41:56.502819+0000","last_fullsized":"2023-10-06T11:41:56.502819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.500296+0000","last_change":"2023-10-06T11:41:55.508163+0000","last_active":"2023-10-06T11:41:56.500296+0000","last_peered":"2023-10-06T11:41:56.500296+0000","last_clean":"2023-10-06T11:41:56.500296+0000","last_became_active":"2023-10-06T11:41:55.507855+0000","last_became_peered":"2023-10-06T11:41:55.507855+0000","last_unstale":"2023-10-06T11:41:56.500296+0000","last_undegraded":"2023-10-06T11:41:56.500296+0000","last_fullsized":"2023-10-06T11:41:56.500296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516585+0000","last_change":"2023-10-06T11:41:56.512837+0000","last_active":"2023-10-06T11:41:56.516585+0000","last_peered":"2023-10-06T11:41:56.516585+0000","last_clean":"2023-10-06T11:41:56.516585+0000","last_became_active":"2023-10-06T11:41:56.511727+0000","last_became_peered":"2023-10-06T11:41:56.511727+0000","last_unstale":"2023-10-06T11:41:56.516585+0000","last_undegraded":"2023-10-06T11:41:56.516585+0000","last_fullsized":"2023-10-06T11:41:56.516585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:19:21.062998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507674+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:41:56.507674+0000","last_peered":"2023-10-06T11:41:56.507674+0000","last_clean":"2023-10-06T11:41:56.507674+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:41:56.507674+0000","last_undegraded":"2023-10-06T11:41:56.507674+0000","last_fullsized":"2023-10-06T11:41:56.507674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.497703+0000","last_change":"2023-10-06T11:41:55.508524+0000","last_active":"2023-10-06T11:41:56.497703+0000","last_peered":"2023-10-06T11:41:56.497703+0000","last_clean":"2023-10-06T11:41:56.497703+0000","last_became_active":"2023-10-06T11:41:55.505590+0000","last_became_peered":"2023-10-06T11:41:55.505590+0000","last_unstale":"2023-10-06T11:41:56.497703+0000","last_undegraded":"2023-10-06T11:41:56.497703+0000","last_fullsized":"2023-10-06T11:41:56.497703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.511544+0000","last_change":"2023-10-06T11:41:56.509540+0000","last_active":"2023-10-06T11:41:56.511544+0000","last_peered":"2023-10-06T11:41:56.511544+0000","last_clean":"2023-10-06T11:41:56.511544+0000","last_became_active":"2023-10-06T11:41:56.509208+0000","last_became_peered":"2023-10-06T11:41:56.509208+0000","last_unstale":"2023-10-06T11:41:56.511544+0000","last_undegraded":"2023-10-06T11:41:56.511544+0000","last_fullsized":"2023-10-06T11:41:56.511544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:27:59.864736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.507693+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:41:56.507693+0000","last_peered":"2023-10-06T11:41:56.507693+0000","last_clean":"2023-10-06T11:41:56.507693+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:41:56.507693+0000","last_undegraded":"2023-10-06T11:41:56.507693+0000","last_fullsized":"2023-10-06T11:41:56.507693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.503179+0000","last_change":"2023-10-06T11:41:55.505392+0000","last_active":"2023-10-06T11:41:56.503179+0000","last_peered":"2023-10-06T11:41:56.503179+0000","last_clean":"2023-10-06T11:41:56.503179+0000","last_became_active":"2023-10-06T11:41:55.505103+0000","last_became_peered":"2023-10-06T11:41:55.505103+0000","last_unstale":"2023-10-06T11:41:56.503179+0000","last_undegraded":"2023-10-06T11:41:56.503179+0000","last_fullsized":"2023-10-06T11:41:56.503179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:26:43.560057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516182+0000","last_change":"2023-10-06T11:41:56.509787+0000","last_active":"2023-10-06T11:41:56.516182+0000","last_peered":"2023-10-06T11:41:56.516182+0000","last_clean":"2023-10-06T11:41:56.516182+0000","last_became_active":"2023-10-06T11:41:56.509495+0000","last_became_peered":"2023-10-06T11:41:56.509495+0000","last_unstale":"2023-10-06T11:41:56.516182+0000","last_undegraded":"2023-10-06T11:41:56.516182+0000","last_fullsized":"2023-10-06T11:41:56.516182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.531881+0000","last_change":"2023-10-06T11:41:55.501475+0000","last_active":"2023-10-06T11:41:56.531881+0000","last_peered":"2023-10-06T11:41:56.531881+0000","last_clean":"2023-10-06T11:41:56.531881+0000","last_became_active":"2023-10-06T11:41:55.500531+0000","last_became_peered":"2023-10-06T11:41:55.500531+0000","last_unstale":"2023-10-06T11:41:56.531881+0000","last_undegraded":"2023-10-06T11:41:56.531881+0000","last_fullsized":"2023-10-06T11:41:56.531881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:58:15.880777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519814+0000","last_change":"2023-10-06T11:41:56.518985+0000","last_active":"2023-10-06T11:41:56.519814+0000","last_peered":"2023-10-06T11:41:56.519814+0000","last_clean":"2023-10-06T11:41:56.519814+0000","last_became_active":"2023-10-06T11:41:56.518656+0000","last_became_peered":"2023-10-06T11:41:56.518656+0000","last_unstale":"2023-10-06T11:41:56.519814+0000","last_undegraded":"2023-10-06T11:41:56.519814+0000","last_fullsized":"2023-10-06T11:41:56.519814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:21.108193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534877+0000","last_change":"2023-10-06T11:41:55.506318+0000","last_active":"2023-10-06T11:41:56.534877+0000","last_peered":"2023-10-06T11:41:56.534877+0000","last_clean":"2023-10-06T11:41:56.534877+0000","last_became_active":"2023-10-06T11:41:55.506045+0000","last_became_peered":"2023-10-06T11:41:55.506045+0000","last_unstale":"2023-10-06T11:41:56.534877+0000","last_undegraded":"2023-10-06T11:41:56.534877+0000","last_fullsized":"2023-10-06T11:41:56.534877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:42:24.563781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520066+0000","last_change":"2023-10-06T11:41:56.515883+0000","last_active":"2023-10-06T11:41:56.520066+0000","last_peered":"2023-10-06T11:41:56.520066+0000","last_clean":"2023-10-06T11:41:56.520066+0000","last_became_active":"2023-10-06T11:41:56.515622+0000","last_became_peered":"2023-10-06T11:41:56.515622+0000","last_unstale":"2023-10-06T11:41:56.520066+0000","last_undegraded":"2023-10-06T11:41:56.520066+0000","last_fullsized":"2023-10-06T11:41:56.520066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:31.056575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.535054+0000","last_change":"2023-10-06T11:41:55.507981+0000","last_active":"2023-10-06T11:41:56.535054+0000","last_peered":"2023-10-06T11:41:56.535054+0000","last_clean":"2023-10-06T11:41:56.535054+0000","last_became_active":"2023-10-06T11:41:55.507665+0000","last_became_peered":"2023-10-06T11:41:55.507665+0000","last_unstale":"2023-10-06T11:41:56.535054+0000","last_undegraded":"2023-10-06T11:41:56.535054+0000","last_fullsized":"2023-10-06T11:41:56.535054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:20.103902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.513856+0000","last_change":"2023-10-06T11:41:56.510088+0000","last_active":"2023-10-06T11:41:56.513856+0000","last_peered":"2023-10-06T11:41:56.513856+0000","last_clean":"2023-10-06T11:41:56.513856+0000","last_became_active":"2023-10-06T11:41:56.509600+0000","last_became_peered":"2023-10-06T11:41:56.509600+0000","last_unstale":"2023-10-06T11:41:56.513856+0000","last_undegraded":"2023-10-06T11:41:56.513856+0000","last_fullsized":"2023-10-06T11:41:56.513856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:18:16.781029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534276+0000","last_change":"2023-10-06T11:41:55.503624+0000","last_active":"2023-10-06T11:41:56.534276+0000","last_peered":"2023-10-06T11:41:56.534276+0000","last_clean":"2023-10-06T11:41:56.534276+0000","last_became_active":"2023-10-06T11:41:55.503318+0000","last_became_peered":"2023-10-06T11:41:55.503318+0000","last_unstale":"2023-10-06T11:41:56.534276+0000","last_undegraded":"2023-10-06T11:41:56.534276+0000","last_fullsized":"2023-10-06T11:41:56.534276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:44.190608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519991+0000","last_change":"2023-10-06T11:41:56.510835+0000","last_active":"2023-10-06T11:41:56.519991+0000","last_peered":"2023-10-06T11:41:56.519991+0000","last_clean":"2023-10-06T11:41:56.519991+0000","last_became_active":"2023-10-06T11:41:56.510547+0000","last_became_peered":"2023-10-06T11:41:56.510547+0000","last_unstale":"2023-10-06T11:41:56.519991+0000","last_undegraded":"2023-10-06T11:41:56.519991+0000","last_fullsized":"2023-10-06T11:41:56.519991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.534863+0000","last_change":"2023-10-06T11:41:55.497178+0000","last_active":"2023-10-06T11:41:56.534863+0000","last_peered":"2023-10-06T11:41:56.534863+0000","last_clean":"2023-10-06T11:41:56.534863+0000","last_became_active":"2023-10-06T11:41:55.495336+0000","last_became_peered":"2023-10-06T11:41:55.495336+0000","last_unstale":"2023-10-06T11:41:56.534863+0000","last_undegraded":"2023-10-06T11:41:56.534863+0000","last_fullsized":"2023-10-06T11:41:56.534863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:54.478477+0000","last_clean_scrub_stamp":"2023-10-06T11:41:54.478477+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:41:19.993711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.519888+0000","last_change":"2023-10-06T11:41:56.511788+0000","last_active":"2023-10-06T11:41:56.519888+0000","last_peered":"2023-10-06T11:41:56.519888+0000","last_clean":"2023-10-06T11:41:56.519888+0000","last_became_active":"2023-10-06T11:41:56.511488+0000","last_became_peered":"2023-10-06T11:41:56.511488+0000","last_unstale":"2023-10-06T11:41:56.519888+0000","last_undegraded":"2023-10-06T11:41:56.519888+0000","last_fullsized":"2023-10-06T11:41:56.519888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.516639+0000","last_change":"2023-10-06T11:41:56.511919+0000","last_active":"2023-10-06T11:41:56.516639+0000","last_peered":"2023-10-06T11:41:56.516639+0000","last_clean":"2023-10-06T11:41:56.516639+0000","last_became_active":"2023-10-06T11:41:56.511644+0000","last_became_peered":"2023-10-06T11:41:56.511644+0000","last_unstale":"2023-10-06T11:41:56.516639+0000","last_undegraded":"2023-10-06T11:41:56.516639+0000","last_fullsized":"2023-10-06T11:41:56.516639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:53.473041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520154+0000","last_change":"2023-10-06T11:41:56.516583+0000","last_active":"2023-10-06T11:41:56.520154+0000","last_peered":"2023-10-06T11:41:56.520154+0000","last_clean":"2023-10-06T11:41:56.520154+0000","last_became_active":"2023-10-06T11:41:56.516241+0000","last_became_peered":"2023-10-06T11:41:56.516241+0000","last_unstale":"2023-10-06T11:41:56.520154+0000","last_undegraded":"2023-10-06T11:41:56.520154+0000","last_fullsized":"2023-10-06T11:41:56.520154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:51:32.340222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520023+0000","last_change":"2023-10-06T11:41:56.514199+0000","last_active":"2023-10-06T11:41:56.520023+0000","last_peered":"2023-10-06T11:41:56.520023+0000","last_clean":"2023-10-06T11:41:56.520023+0000","last_became_active":"2023-10-06T11:41:56.513961+0000","last_became_peered":"2023-10-06T11:41:56.513961+0000","last_unstale":"2023-10-06T11:41:56.520023+0000","last_undegraded":"2023-10-06T11:41:56.520023+0000","last_fullsized":"2023-10-06T11:41:56.520023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:37.285579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520267+0000","last_change":"2023-10-06T11:41:56.518788+0000","last_active":"2023-10-06T11:41:56.520267+0000","last_peered":"2023-10-06T11:41:56.520267+0000","last_clean":"2023-10-06T11:41:56.520267+0000","last_became_active":"2023-10-06T11:41:56.516640+0000","last_became_peered":"2023-10-06T11:41:56.516640+0000","last_unstale":"2023-10-06T11:41:56.520267+0000","last_undegraded":"2023-10-06T11:41:56.520267+0000","last_fullsized":"2023-10-06T11:41:56.520267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:29:14.957485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T11:41:56.520067+0000","last_change":"2023-10-06T11:41:56.516139+0000","last_active":"2023-10-06T11:41:56.520067+0000","last_peered":"2023-10-06T11:41:56.520067+0000","last_clean":"2023-10-06T11:41:56.520067+0000","last_became_active":"2023-10-06T11:41:56.514921+0000","last_became_peered":"2023-10-06T11:41:56.514921+0000","last_unstale":"2023-10-06T11:41:56.520067+0000","last_undegraded":"2023-10-06T11:41:56.520067+0000","last_fullsized":"2023-10-06T11:41:56.520067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:55.482768+0000","last_clean_scrub_stamp":"2023-10-06T11:41:55.482768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:57:33.392376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":671324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4649999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0229999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9600000000000009}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.6189999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":78316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6849999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8239999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.9279999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":79828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:05.719 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T11:42:05.719 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T11:42:05.720 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T11:42:05.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T11:42:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.869+0000 7faf3e5fd700 1 Processor -- start 2023-10-06T11:42:05.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 -- start start 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf38156ef0 0x7faf381572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf3815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380b1680 con 0x7faf3806d520 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3815ac40 con 0x7faf38156ef0 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf3e5fd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3806f540 con 0x7faf380b1c60 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.870+0000 7faf37fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf38156ef0 0x7faf381572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf38156ef0 0x7faf381572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43428/0 (socket says 172.21.15.16:43428) 2023-10-06T11:42:05.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 -- 172.21.15.16:0/2611057759 learned_addr learned my addr 172.21.15.16:0/2611057759 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf377fe700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf3815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 -- 172.21.15.16:0/2611057759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 msgr2=0x7faf3815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf3815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 -- 172.21.15.16:0/2611057759 >> v1:172.21.15.177:6789/0 conn(0x7faf3806d520 legacy=0x7faf380b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 -- 172.21.15.16:0/2611057759 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380b21b0 con 0x7faf38156ef0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf37fff700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf381572c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf28009990 tx=0x7faf28009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d18fa3eb146f2b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf36ffd700 1 -- 172.21.15.16:0/2611057759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf28005c10 con 0x7faf38156ef0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf36ffd700 1 -- 172.21.15.16:0/2611057759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf28005d70 con 0x7faf38156ef0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.871+0000 7faf36ffd700 1 -- 172.21.15.16:0/2611057759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800b8a0 con 0x7faf38156ef0 2023-10-06T11:42:05.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 -- 172.21.15.16:0/2611057759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 msgr2=0x7faf381572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf381572c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf28009990 tx=0x7faf28009220 comp rx=0 tx=0).stop 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 -- 172.21.15.16:0/2611057759 shutdown_connections 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf381572c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/2611057759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf3815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 -- 172.21.15.16:0/2611057759 >> 172.21.15.16:0/2611057759 conn(0x7faf38067e60 msgr2=0x7faf38068260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 -- 172.21.15.16:0/2611057759 shutdown_connections 2023-10-06T11:42:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.872+0000 7faf3e5fd700 1 -- 172.21.15.16:0/2611057759 wait complete. 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.873+0000 7faf3e5fd700 1 Processor -- start 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.873+0000 7faf3e5fd700 1 -- start start 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf3806d520 0x7faf3815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf381698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c7740 con 0x7faf38156ef0 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c7990 con 0x7faf3806d520 2023-10-06T11:42:05.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c7c70 con 0x7faf380b1c60 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf37fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf3806d520 0x7faf3815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43444/0 (socket says 172.21.15.16:43444) 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 -- 172.21.15.16:0/4181962148 learned_addr learned my addr 172.21.15.16:0/4181962148 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf377fe700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf381698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 -- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 msgr2=0x7faf381698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf381698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 -- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf3806d520 msgr2=0x7faf3815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf3806d520 0x7faf3815eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:05.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 -- 172.21.15.16:0/4181962148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28009480 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3cb9a700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf3000c8e0 tx=0x7faf30001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30012630 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30012790 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30012b50 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.874+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380c7f50 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.875+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3815f410 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.875+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3804c380 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.876+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf3001a3a0 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.876+0000 7faf1ffff700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.876+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7faf300847d0 con 0x7faf38156ef0 2023-10-06T11:42:05.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.876+0000 7faf37fff700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.877+0000 7faf37fff700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 0x7faf240693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:05.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:05.880+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf30055370 con 0x7faf38156ef0 2023-10-06T11:42:06.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.118+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faf3806ef80 con 0x7faf38156ef0 2023-10-06T11:42:06.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.118+0000 7faf1ffff700 1 -- 172.21.15.16:0/4181962148 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7faf30054f00 con 0x7faf38156ef0 2023-10-06T11:42:06.120 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:06.121 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 msgr2=0x7faf240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 0x7faf240693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28016000 comp rx=0 tx=0).stop 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 msgr2=0x7faf38146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7faf3000c8e0 tx=0x7faf30001710 comp rx=0 tx=0).stop 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 shutdown_connections 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf38156ef0 0x7faf38146bc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7faf24066f30 0x7faf240693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7faf3806d520 0x7faf3815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 --2- 172.21.15.16:0/4181962148 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7faf380b1c60 0x7faf381698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 >> 172.21.15.16:0/4181962148 conn(0x7faf38067e60 msgr2=0x7faf38157350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 shutdown_connections 2023-10-06T11:42:06.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.120+0000 7faf3e5fd700 1 -- 172.21.15.16:0/4181962148 wait complete. 2023-10-06T11:42:06.134 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T11:42:06.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.251+0000 7f4377eaa700 1 Processor -- start 2023-10-06T11:42:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.251+0000 7f4377eaa700 1 -- start start 2023-10-06T11:42:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4377eaa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4377eaa700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f43700cf450 0x7f43700c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4377eaa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700cdb10 con 0x7f43700cd6a0 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4377eaa700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700b3110 con 0x7f43700cf450 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4377eaa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43700cf8d0 con 0x7f437015cc20 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4375445700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f43700cd6a0 0x7f43700ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57068/0 (socket says 172.21.15.16:57068) 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4375445700 1 -- 172.21.15.16:0/4115610784 learned_addr learned my addr 172.21.15.16:0/4115610784 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4376447700 1 --2- 172.21.15.16:0/4115610784 >> v2:172.21.15.177:3300/0 conn(0x7f43700cf450 0x7f43700c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.252+0000 7f4375c46700 1 --2- 172.21.15.16:0/4115610784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693072058 0 0) 0x7f43700cdb10 con 0x7f43700cd6a0 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4358003570 con 0x7f43700cd6a0 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4375c46700 1 -- 172.21.15.16:0/4115610784 >> v2:172.21.15.177:3300/0 conn(0x7f43700cf450 msgr2=0x7f43700c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4375c46700 1 --2- 172.21.15.16:0/4115610784 >> v2:172.21.15.177:3300/0 conn(0x7f43700cf450 0x7f43700c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4375c46700 1 -- 172.21.15.16:0/4115610784 >> v1:172.21.15.16:6789/0 conn(0x7f43700cd6a0 legacy=0x7f43700ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622355732 0 0) 0x7f4358003570 con 0x7f43700cd6a0 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4375c46700 1 -- 172.21.15.16:0/4115610784 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700ccb20 con 0x7f437015cc20 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4375c46700 1 --2- 172.21.15.16:0/4115610784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700cd160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43680098e0 tx=0x7f43680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0bef78010113ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364001470 con 0x7f437015cc20 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4368011420 con 0x7f437015cc20 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.253+0000 7f4374c44700 1 -- 172.21.15.16:0/4115610784 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43680117c0 con 0x7f437015cc20 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 -- 172.21.15.16:0/4115610784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 msgr2=0x7f43700cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 --2- 172.21.15.16:0/4115610784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700cd160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43680098e0 tx=0x7f43680091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 -- 172.21.15.16:0/4115610784 shutdown_connections 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 --2- 172.21.15.16:0/4115610784 >> v2:172.21.15.177:3300/0 conn(0x7f43700cf450 0x7f43700c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 --2- 172.21.15.16:0/4115610784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700cd160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 -- 172.21.15.16:0/4115610784 >> 172.21.15.16:0/4115610784 conn(0x7f4370099b40 msgr2=0x7f437006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 -- 172.21.15.16:0/4115610784 shutdown_connections 2023-10-06T11:42:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.254+0000 7f4377eaa700 1 -- 172.21.15.16:0/4115610784 wait complete. 2023-10-06T11:42:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.255+0000 7f4377eaa700 1 Processor -- start 2023-10-06T11:42:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.255+0000 7f4377eaa700 1 -- start start 2023-10-06T11:42:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43700cd6a0 0x7f43700b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437015fa40 con 0x7f43700cf450 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437015ed30 con 0x7f43700cd6a0 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b6260 con 0x7f437015cc20 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43458/0 (socket says 172.21.15.16:43458) 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 -- 172.21.15.16:0/2591706392 learned_addr learned my addr 172.21.15.16:0/2591706392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4376447700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375c46700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43700cd6a0 0x7f43700b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 -- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 msgr2=0x7f43700b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 -- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43700cd6a0 msgr2=0x7f43700b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43700cd6a0 0x7f43700b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 -- 172.21.15.16:0/2591706392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368009420 con 0x7f43700cf450 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4375445700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f436400cb90 tx=0x7f4364001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364007150 con 0x7f43700cf450 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640072d0 con 0x7f43700cf450 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364007690 con 0x7f43700cf450 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.256+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43700b4640 con 0x7f43700cf450 2023-10-06T11:42:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.257+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437015f240 con 0x7f43700cf450 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.258+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437004c380 con 0x7f43700cf450 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.258+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4364004a40 con 0x7f43700cf450 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.258+0000 7f436dffb700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 0x7f435c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.258+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4364083d90 con 0x7f43700cf450 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.258+0000 7f4375c46700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 0x7f435c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.259+0000 7f4375c46700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 0x7f435c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43680093f0 tx=0x7f4368016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.262+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4364050f20 con 0x7f43700cf450 2023-10-06T11:42:06.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.457+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f437006fa90 con 0x7f43700cf450 2023-10-06T11:42:06.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.458+0000 7f436dffb700 1 -- 172.21.15.16:0/2591706392 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f4364018020 con 0x7f43700cf450 2023-10-06T11:42:06.461 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:06.461 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 msgr2=0x7f435c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 0x7f435c0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f43680093f0 tx=0x7f4368016000 comp rx=0 tx=0).stop 2023-10-06T11:42:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 msgr2=0x7f4370153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f436400cb90 tx=0x7f4364001ab0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 shutdown_connections 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f435c066f30 0x7f435c0693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f437015cc20 0x7f43700b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43700cd6a0 0x7f43700b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 --2- 172.21.15.16:0/2591706392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43700cf450 0x7f4370153640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 >> 172.21.15.16:0/2591706392 conn(0x7f4370099b40 msgr2=0x7f437015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 shutdown_connections 2023-10-06T11:42:06.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.461+0000 7f4377eaa700 1 -- 172.21.15.16:0/2591706392 wait complete. 2023-10-06T11:42:06.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:06.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.595+0000 7fb9939c9700 1 Processor -- start 2023-10-06T11:42:06.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 -- start start 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb98c1570d0 0x7fb98c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c0b3000 0x7fb98c06c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98c06bee0 con 0x7fb98c1574a0 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c06a7f0 con 0x7fb98c1570d0 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb9939c9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c15a130 con 0x7fb98c0b3000 2023-10-06T11:42:06.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb991765700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb98c1570d0 0x7fb98c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.596+0000 7fb991765700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb98c1570d0 0x7fb98c0b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43466/0 (socket says 172.21.15.16:43466) 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 -- 172.21.15.16:0/2813281792 learned_addr learned my addr 172.21.15.16:0/2813281792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb990f64700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c0b3000 0x7fb98c06c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 -- 172.21.15.16:0/2813281792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c0b3000 msgr2=0x7fb98c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c0b3000 0x7fb98c06c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 -- 172.21.15.16:0/2813281792 >> v1:172.21.15.177:6789/0 conn(0x7fb98c1574a0 legacy=0x7fb98c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 -- 172.21.15.16:0/2813281792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c15a410 con 0x7fb98c1570d0 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb991765700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c1570d0 0x7fb98c0b2ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb984009f20 tx=0x7fb984009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3825dafb9a00bc17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb98bfff700 1 -- 172.21.15.16:0/2813281792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400e070 con 0x7fb98c1570d0 2023-10-06T11:42:06.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb98bfff700 1 -- 172.21.15.16:0/2813281792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98400ba40 con 0x7fb98c1570d0 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.597+0000 7fb98bfff700 1 -- 172.21.15.16:0/2813281792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840092e0 con 0x7fb98c1570d0 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 -- 172.21.15.16:0/2813281792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c1570d0 msgr2=0x7fb98c0b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c1570d0 0x7fb98c0b2ac0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb984009f20 tx=0x7fb984009840 comp rx=0 tx=0).stop 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 -- 172.21.15.16:0/2813281792 shutdown_connections 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c1570d0 0x7fb98c0b2ac0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2813281792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c0b3000 0x7fb98c06c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 -- 172.21.15.16:0/2813281792 >> 172.21.15.16:0/2813281792 conn(0x7fb98c068030 msgr2=0x7fb98c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 -- 172.21.15.16:0/2813281792 shutdown_connections 2023-10-06T11:42:06.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.598+0000 7fb9939c9700 1 -- 172.21.15.16:0/2813281792 wait complete. 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.599+0000 7fb9939c9700 1 Processor -- start 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.599+0000 7fb9939c9700 1 -- start start 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb98c1570d0 0x7fb98c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c1574a0 0x7fb98c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c151f80 con 0x7fb98c0b3000 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c1510b0 con 0x7fb98c1570d0 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c151300 con 0x7fb98c1574a0 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43478/0 (socket says 172.21.15.16:43478) 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 -- 172.21.15.16:0/2820778251 learned_addr learned my addr 172.21.15.16:0/2820778251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:06.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb990f64700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb98c1570d0 0x7fb98c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991f66700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c1574a0 0x7fb98c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 -- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c1574a0 msgr2=0x7fb98c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c1574a0 0x7fb98c14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 -- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb98c1570d0 msgr2=0x7fb98c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb98c1570d0 0x7fb98c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 -- 172.21.15.16:0/2820778251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984009ae0 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb991765700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb98400b6c0 tx=0x7fb984017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb984012aa0 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984012c20 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98401d420 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c1515e0 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.600+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c151ba0 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.601+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c04c380 con 0x7fb98c0b3000 2023-10-06T11:42:06.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.602+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb98401d830 con 0x7fb98c0b3000 2023-10-06T11:42:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.602+0000 7fb9897fa700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 0x7fb978069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.602+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb984085d90 con 0x7fb98c0b3000 2023-10-06T11:42:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.602+0000 7fb990f64700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 0x7fb978069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.603+0000 7fb990f64700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 0x7fb978069410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb980006fd0 tx=0x7fb980008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.606+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb984052ea0 con 0x7fb98c0b3000 2023-10-06T11:42:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.799+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb98c0b0330 con 0x7fb98c0b3000 2023-10-06T11:42:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.799+0000 7fb9897fa700 1 -- 172.21.15.16:0/2820778251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb9840564c0 con 0x7fb98c0b3000 2023-10-06T11:42:06.803 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 msgr2=0x7fb978069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 0x7fb978069410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb980006fd0 tx=0x7fb980008040 comp rx=0 tx=0).stop 2023-10-06T11:42:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 msgr2=0x7fb98c152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb98400b6c0 tx=0x7fb984017000 comp rx=0 tx=0).stop 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 shutdown_connections 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb98c0b3000 0x7fb98c152e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb978066f60 0x7fb978069410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb98c1574a0 0x7fb98c14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 --2- 172.21.15.16:0/2820778251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb98c1570d0 0x7fb98c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 >> 172.21.15.16:0/2820778251 conn(0x7fb98c068030 msgr2=0x7fb98c15d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 shutdown_connections 2023-10-06T11:42:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.801+0000 7fb9939c9700 1 -- 172.21.15.16:0/2820778251 wait complete. 2023-10-06T11:42:06.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:06.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.936+0000 7f6651f5d700 1 Processor -- start 2023-10-06T11:42:06.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.936+0000 7f6651f5d700 1 -- start start 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f6651f5d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f664c0cb660 0x7f664c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f6651f5d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c13c9f0 con 0x7f664c0cc7d0 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f6651f5d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c06dff0 con 0x7f664c0cb290 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f6651f5d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15e270 con 0x7f664c0cb660 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f664affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f664c0cc7d0 0x7f664c15d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57076/0 (socket says 172.21.15.16:57076) 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f664affd700 1 -- 172.21.15.16:0/3923577186 learned_addr learned my addr 172.21.15.16:0/3923577186 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.937+0000 7f664bfff700 1 --2- 172.21.15.16:0/3923577186 >> v2:172.21.15.177:3300/0 conn(0x7f664c0cb660 0x7f664c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418069095 0 0) 0x7f6640001110 con 0x7f664c0cc7d0 2023-10-06T11:42:06.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003570 con 0x7f664c0cc7d0 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150864014 0 0) 0x7f664c06dff0 con 0x7f664c0cb290 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6640001110 con 0x7f664c0cb290 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664bfff700 1 -- 172.21.15.16:0/3923577186 >> v1:172.21.15.177:6789/0 conn(0x7f664c0cb290 legacy=0x7f664c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664bfff700 1 -- 172.21.15.16:0/3923577186 >> v1:172.21.15.16:6789/0 conn(0x7f664c0cc7d0 legacy=0x7f664c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664bfff700 1 -- 172.21.15.16:0/3923577186 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c15ae30 con 0x7f664c0cb660 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664bfff700 1 --2- 172.21.15.16:0/3923577186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cb660 0x7f664c15dc10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6644009e60 tx=0x7f6644009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=897c08efb66f7c4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400e070 con 0x7f664c0cb660 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400bab0 con 0x7f664c0cb660 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.938+0000 7f664a7fc700 1 -- 172.21.15.16:0/3923577186 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400be70 con 0x7f664c0cb660 2023-10-06T11:42:06.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 -- 172.21.15.16:0/3923577186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cb660 msgr2=0x7f664c15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 --2- 172.21.15.16:0/3923577186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cb660 0x7f664c15dc10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6644009e60 tx=0x7f6644009780 comp rx=0 tx=0).stop 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 -- 172.21.15.16:0/3923577186 shutdown_connections 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 --2- 172.21.15.16:0/3923577186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cb660 0x7f664c15dc10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 -- 172.21.15.16:0/3923577186 >> 172.21.15.16:0/3923577186 conn(0x7f664c067c90 msgr2=0x7f664c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 -- 172.21.15.16:0/3923577186 shutdown_connections 2023-10-06T11:42:06.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.939+0000 7f6651f5d700 1 -- 172.21.15.16:0/3923577186 wait complete. 2023-10-06T11:42:06.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.940+0000 7f6651f5d700 1 Processor -- start 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.940+0000 7f6651f5d700 1 -- start start 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f664c0cb290 0x7f664c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cc7d0 0x7f664c13aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0c55a0 con 0x7f664c0cb660 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c16a920 con 0x7f664c0cc7d0 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c16ac00 con 0x7f664c0cb290 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43488/0 (socket says 172.21.15.16:43488) 2023-10-06T11:42:06.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 -- 172.21.15.16:0/1492343648 learned_addr learned my addr 172.21.15.16:0/1492343648 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664bfff700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cc7d0 0x7f664c13aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664b7fe700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f664c0cb290 0x7f664c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 -- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f664c0cb290 msgr2=0x7f664c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f664c0cb290 0x7f664c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 -- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cc7d0 msgr2=0x7f664c13aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cc7d0 0x7f664c13aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 -- 172.21.15.16:0/1492343648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644009a20 con 0x7f664c0cb660 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f664affd700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f664000c790 tx=0x7f6640007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640005b30 con 0x7f664c0cb660 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6640005c90 con 0x7f664c0cb660 2023-10-06T11:42:06.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640007300 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c0c5780 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.941+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c0c82a0 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.942+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c0c5ca0 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.943+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f664001b3b0 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.943+0000 7f66337fe700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 0x7f66380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.943+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6640083b10 con 0x7f664c0cb660 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.943+0000 7f664b7fe700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 0x7f66380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:06.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.944+0000 7f664b7fe700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 0x7f66380693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f664c142c50 tx=0x7f663c007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:06.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:06.947+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f664c0c5ca0 con 0x7f664c0cb660 2023-10-06T11:42:07.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.139+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f664c13c9f0 con 0x7f664c0cb660 2023-10-06T11:42:07.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.140+0000 7f66337fe700 1 -- 172.21.15.16:0/1492343648 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f664c13c9f0 con 0x7f664c0cb660 2023-10-06T11:42:07.142 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 msgr2=0x7f66380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 0x7f66380693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f664c142c50 tx=0x7f663c007d50 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 msgr2=0x7f664c169e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f664000c790 tx=0x7f6640007e40 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 shutdown_connections 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6638066f30 0x7f66380693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f664c0cc7d0 0x7f664c13aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f664c0cb290 0x7f664c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 --2- 172.21.15.16:0/1492343648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f664c0cb660 0x7f664c169e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 >> 172.21.15.16:0/1492343648 conn(0x7f664c067c90 msgr2=0x7f664c13bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 shutdown_connections 2023-10-06T11:42:07.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.142+0000 7f6651f5d700 1 -- 172.21.15.16:0/1492343648 wait complete. 2023-10-06T11:42:07.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.275+0000 7f37f5012700 1 Processor -- start 2023-10-06T11:42:07.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.275+0000 7f37f5012700 1 -- start start 2023-10-06T11:42:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37f5012700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37f00cda10 0x7f37f00b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37f5012700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f0159b10 0x7f37f00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37f5012700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f00cead0 con 0x7f37f0159740 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37f5012700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f00affc0 con 0x7f37f00cda10 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37f5012700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00cdec0 con 0x7f37f0159b10 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37ee59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37f00cda10 0x7f37f00b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37eed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f37f0159740 0x7f37f00b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57090/0 (socket says 172.21.15.16:57090) 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37eed9d700 1 -- 172.21.15.16:0/1142842352 learned_addr learned my addr 172.21.15.16:0/1142842352 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.276+0000 7f37ee59c700 1 -- 172.21.15.16:0/1142842352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f0159b10 msgr2=0x7f37f00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37ee59c700 1 --2- 172.21.15.16:0/1142842352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f0159b10 0x7f37f00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37ee59c700 1 -- 172.21.15.16:0/1142842352 >> v1:172.21.15.16:6789/0 conn(0x7f37f0159740 legacy=0x7f37f00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37ee59c700 1 -- 172.21.15.16:0/1142842352 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00ce1a0 con 0x7f37f00cda10 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37ee59c700 1 --2- 172.21.15.16:0/1142842352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00cda10 0x7f37f00b40a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37d80098e0 tx=0x7f37d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f5eafc1255986a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37edd9b700 1 -- 172.21.15.16:0/1142842352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d8009e80 con 0x7f37f00cda10 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37edd9b700 1 -- 172.21.15.16:0/1142842352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d800b760 con 0x7f37f00cda10 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37edd9b700 1 -- 172.21.15.16:0/1142842352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d800bae0 con 0x7f37f00cda10 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 -- 172.21.15.16:0/1142842352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00cda10 msgr2=0x7f37f00b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 --2- 172.21.15.16:0/1142842352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00cda10 0x7f37f00b40a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37d80098e0 tx=0x7f37d80091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 -- 172.21.15.16:0/1142842352 shutdown_connections 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 --2- 172.21.15.16:0/1142842352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f0159b10 0x7f37f00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 --2- 172.21.15.16:0/1142842352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f00cda10 0x7f37f00b40a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 -- 172.21.15.16:0/1142842352 >> 172.21.15.16:0/1142842352 conn(0x7f37f0099a40 msgr2=0x7f37f006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 -- 172.21.15.16:0/1142842352 shutdown_connections 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.277+0000 7f37f5012700 1 -- 172.21.15.16:0/1142842352 wait complete. 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 Processor -- start 2023-10-06T11:42:07.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 -- start start 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 0x7f37f016a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f0159740 0x7f37f015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f016acb0 con 0x7f37f0159b10 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f00b14c0 con 0x7f37f0159740 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37f5012700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f009d910 con 0x7f37f00cda10 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37ef59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.279+0000 7f37ef59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43494/0 (socket says 172.21.15.16:43494) 2023-10-06T11:42:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 -- 172.21.15.16:0/2025711026 learned_addr learned my addr 172.21.15.16:0/2025711026 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37eed9d700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 0x7f37f016a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ee59c700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f0159740 0x7f37f015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 -- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 msgr2=0x7f37f016a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 0x7f37f016a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 -- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f0159740 msgr2=0x7f37f015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f0159740 0x7f37f015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 -- 172.21.15.16:0/2025711026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d8009420 con 0x7f37f0159b10 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37eed9d700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 0x7f37f016a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37ef59e700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e000bc30 tx=0x7f37e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00092e0 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e000c7a0 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e000cb80 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f009dbf0 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.280+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f00b1150 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.281+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f00b1630 con 0x7f37f0159b10 2023-10-06T11:42:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.281+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37e0012070 con 0x7f37f0159b10 2023-10-06T11:42:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.282+0000 7f37e6ffd700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 0x7f37d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.282+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f37e00843a0 con 0x7f37f0159b10 2023-10-06T11:42:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.282+0000 7f37eed9d700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 0x7f37d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.282+0000 7f37eed9d700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 0x7f37d00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007fd0 tx=0x7f37e8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.286+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37e00514b0 con 0x7f37f0159b10 2023-10-06T11:42:07.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.478+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37f004c380 con 0x7f37f0159b10 2023-10-06T11:42:07.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.478+0000 7f37e6ffd700 1 -- 172.21.15.16:0/2025711026 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37e0054ad0 con 0x7f37f0159b10 2023-10-06T11:42:07.481 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:07.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 msgr2=0x7f37d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 0x7f37d00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f37e8007fd0 tx=0x7f37e8009040 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 msgr2=0x7f37f015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37e000bc30 tx=0x7f37e0009840 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 shutdown_connections 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37f0159b10 0x7f37f015c800 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37d0066f30 0x7f37d00693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37f00cda10 0x7f37f016a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 --2- 172.21.15.16:0/2025711026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37f0159740 0x7f37f015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 >> 172.21.15.16:0/2025711026 conn(0x7f37f0099a40 msgr2=0x7f37f0156650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 shutdown_connections 2023-10-06T11:42:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.480+0000 7f37f5012700 1 -- 172.21.15.16:0/2025711026 wait complete. 2023-10-06T11:42:07.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.615+0000 7fb8663b8700 1 Processor -- start 2023-10-06T11:42:07.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.615+0000 7fb8663b8700 1 -- start start 2023-10-06T11:42:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.615+0000 7fb8663b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb860159740 0x7fb8600b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb8663b8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb860159b10 0x7fb8600cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb8663b8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8600cead0 con 0x7fb8600cda10 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb8663b8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8600affc0 con 0x7fb860159740 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb8663b8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8600cdec0 con 0x7fb860159b10 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb860159740 0x7fb8600b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb860159740 0x7fb8600b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43506/0 (socket says 172.21.15.16:43506) 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 -- 172.21.15.16:0/789841568 learned_addr learned my addr 172.21.15.16:0/789841568 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb864955700 1 --2- 172.21.15.16:0/789841568 >> v2:172.21.15.177:3300/0 conn(0x7fb860159b10 0x7fb8600cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 -- 172.21.15.16:0/789841568 >> v2:172.21.15.177:3300/0 conn(0x7fb860159b10 msgr2=0x7fb8600cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 --2- 172.21.15.16:0/789841568 >> v2:172.21.15.177:3300/0 conn(0x7fb860159b10 0x7fb8600cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 -- 172.21.15.16:0/789841568 >> v1:172.21.15.177:6789/0 conn(0x7fb8600cda10 legacy=0x7fb8600b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 -- 172.21.15.16:0/789841568 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8600ce1a0 con 0x7fb860159740 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85ffff700 1 --2- 172.21.15.16:0/789841568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb860159740 0x7fb8600b3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b3240 tx=0x7fb850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331e26f3b4a6d644 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85effd700 1 -- 172.21.15.16:0/789841568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb850009e70 con 0x7fb860159740 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85effd700 1 -- 172.21.15.16:0/789841568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85000b860 con 0x7fb860159740 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.616+0000 7fb85effd700 1 -- 172.21.15.16:0/789841568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85000bc00 con 0x7fb860159740 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 -- 172.21.15.16:0/789841568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb860159740 msgr2=0x7fb8600b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 --2- 172.21.15.16:0/789841568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb860159740 0x7fb8600b3990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b3240 tx=0x7fb850009250 comp rx=0 tx=0).stop 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 -- 172.21.15.16:0/789841568 shutdown_connections 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 --2- 172.21.15.16:0/789841568 >> v2:172.21.15.177:3300/0 conn(0x7fb860159b10 0x7fb8600cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 --2- 172.21.15.16:0/789841568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb860159740 0x7fb8600b3990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 -- 172.21.15.16:0/789841568 >> 172.21.15.16:0/789841568 conn(0x7fb860099a40 msgr2=0x7fb86006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 -- 172.21.15.16:0/789841568 shutdown_connections 2023-10-06T11:42:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.617+0000 7fb8663b8700 1 -- 172.21.15.16:0/789841568 wait complete. 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.618+0000 7fb8663b8700 1 Processor -- start 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.618+0000 7fb8663b8700 1 -- start start 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb860159740 0x7fb860155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb860159b10 0x7fb86009bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015a590 con 0x7fb8600cda10 2023-10-06T11:42:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015a870 con 0x7fb860159b10 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb8663b8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015ab50 con 0x7fb860159740 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43516/0 (socket says 172.21.15.16:43516) 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 -- 172.21.15.16:0/2685818959 learned_addr learned my addr 172.21.15.16:0/2685818959 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb864955700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb860159b10 0x7fb86009bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85f7fe700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb860159740 0x7fb860155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 -- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb860159740 msgr2=0x7fb860155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb860159740 0x7fb860155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 -- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb860159b10 msgr2=0x7fb86009bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb860159b10 0x7fb86009bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.619+0000 7fb85ffff700 1 -- 172.21.15.16:0/2685818959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8500094f0 con 0x7fb8600cda10 2023-10-06T11:42:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb85ffff700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb86006d350 tx=0x7fb850017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85000bbe0 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb850011420 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86015ae30 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8500117e0 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.620+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8601699f0 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.621+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86004c380 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.621+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb850011bd0 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.621+0000 7fb847fff700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 0x7fb84c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.622+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb850085a90 con 0x7fb8600cda10 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.622+0000 7fb85f7fe700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 0x7fb84c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.622+0000 7fb85f7fe700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 0x7fb84c069410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b5a90 tx=0x7fb854007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.626+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb850052be0 con 0x7fb8600cda10 2023-10-06T11:42:07.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.819+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8600d0470 con 0x7fb8600cda10 2023-10-06T11:42:07.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.820+0000 7fb847fff700 1 -- 172.21.15.16:0/2685818959 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8500561c0 con 0x7fb8600cda10 2023-10-06T11:42:07.822 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 msgr2=0x7fb84c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 0x7fb84c069410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb8600b5a90 tx=0x7fb854007ec0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 msgr2=0x7fb860159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb86006d350 tx=0x7fb850017000 comp rx=0 tx=0).stop 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 shutdown_connections 2023-10-06T11:42:07.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8600cda10 0x7fb860159fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb84c066f60 0x7fb84c069410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb860159b10 0x7fb86009bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 --2- 172.21.15.16:0/2685818959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb860159740 0x7fb860155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 >> 172.21.15.16:0/2685818959 conn(0x7fb860099a40 msgr2=0x7fb86006f560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 shutdown_connections 2023-10-06T11:42:07.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.822+0000 7fb8663b8700 1 -- 172.21.15.16:0/2685818959 wait complete. 2023-10-06T11:42:07.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.954+0000 7fd90cb4e700 1 Processor -- start 2023-10-06T11:42:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.954+0000 7fd90cb4e700 1 -- start start 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080b3110 con 0x7fd90815cc20 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080cdb10 con 0x7fd9080cf280 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90cb4e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080ccb20 con 0x7fd9080cd6a0 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43524/0 (socket says 172.21.15.16:43524) 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.955+0000 7fd90659c700 1 -- 172.21.15.16:0/4140460842 learned_addr learned my addr 172.21.15.16:0/4140460842 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd905d9b700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd906d9d700 1 --2- 172.21.15.16:0/4140460842 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 -- 172.21.15.16:0/4140460842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 msgr2=0x7fd9080ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 -- 172.21.15.16:0/4140460842 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cf280 msgr2=0x7fd90815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:07.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 --2- 172.21.15.16:0/4140460842 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 -- 172.21.15.16:0/4140460842 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080cbf10 con 0x7fd90815cc20 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90659c700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9080b6760 tx=0x7fd900009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a68c5759f568dfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90559a700 1 -- 172.21.15.16:0/4140460842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000e070 con 0x7fd90815cc20 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90559a700 1 -- 172.21.15.16:0/4140460842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000ba50 con 0x7fd90815cc20 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90559a700 1 -- 172.21.15.16:0/4140460842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9000092e0 con 0x7fd90815cc20 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90cb4e700 1 -- 172.21.15.16:0/4140460842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 msgr2=0x7fd9080cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.956+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9080b6760 tx=0x7fd900009840 comp rx=0 tx=0).stop 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 -- 172.21.15.16:0/4140460842 shutdown_connections 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/4140460842 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/4140460842 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 -- 172.21.15.16:0/4140460842 >> 172.21.15.16:0/4140460842 conn(0x7fd908099b40 msgr2=0x7fd90806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 -- 172.21.15.16:0/4140460842 shutdown_connections 2023-10-06T11:42:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.957+0000 7fd90cb4e700 1 -- 172.21.15.16:0/4140460842 wait complete. 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 Processor -- start 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 -- start start 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd90814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90806e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90809e180 con 0x7fd9080cf280 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c8e80 con 0x7fd90815cc20 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd90cb4e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90814e790 con 0x7fd9080cd6a0 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd905d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.958+0000 7fd905d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43526/0 (socket says 172.21.15.16:43526) 2023-10-06T11:42:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 -- 172.21.15.16:0/1144409661 learned_addr learned my addr 172.21.15.16:0/1144409661 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd90659c700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd90814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd906d9d700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90806e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 -- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 msgr2=0x7fd90814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd90814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 -- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 msgr2=0x7fd90806e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90806e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 -- 172.21.15.16:0/1144409661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009ae0 con 0x7fd9080cf280 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd906d9d700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90806e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd905d9b700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000d860 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00125f0 con 0x7fd9080cf280 2023-10-06T11:42:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0012750 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90809d8e0 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0012b10 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.959+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90809dea0 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.960+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9080c8ff0 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.960+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8f001e460 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.961+0000 7fd8fe7fc700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.961+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8f0084c30 con 0x7fd9080cf280 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.961+0000 7fd90659c700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.961+0000 7fd90659c700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9080b6760 tx=0x7fd900018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:07.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:07.965+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8f0051d40 con 0x7fd9080cf280 2023-10-06T11:42:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.202+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd90804c380 con 0x7fd9080cf280 2023-10-06T11:42:08.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.202+0000 7fd8fe7fc700 1 -- 172.21.15.16:0/1144409661 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fd8f0055360 con 0x7fd9080cf280 2023-10-06T11:42:08.205 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:08.205 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.204+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 msgr2=0x7fd8e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.204+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd9080b6760 tx=0x7fd900018000 comp rx=0 tx=0).stop 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 msgr2=0x7fd90806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000d860 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 shutdown_connections 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90806e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cd6a0 0x7fd90814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 --2- 172.21.15.16:0/1144409661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cf280 0x7fd90806dc90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 >> 172.21.15.16:0/1144409661 conn(0x7fd908099b40 msgr2=0x7fd908159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 shutdown_connections 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.205+0000 7fd90cb4e700 1 -- 172.21.15.16:0/1144409661 wait complete. 2023-10-06T11:42:08.208 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:08.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.337+0000 7f5acd83f700 1 Processor -- start 2023-10-06T11:42:08.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.337+0000 7f5acd83f700 1 -- start start 2023-10-06T11:42:08.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5acd83f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ac815cc20 0x7f5ac80cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5acd83f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5acd83f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac80cdb10 con 0x7f5ac80cf280 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5acd83f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac80b3110 con 0x7f5ac815cc20 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5acd83f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80ccb20 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac6ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ac815cc20 0x7f5ac80cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac67fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac77fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f5ac80cf280 0x7f5ac815d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:40144/0 (socket says 172.21.15.16:40144) 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac77fe700 1 -- 172.21.15.16:0/1911077135 learned_addr learned my addr 172.21.15.16:0/1911077135 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac67fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:55974/0 (socket says 172.21.15.16:55974) 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac67fc700 1 -- 172.21.15.16:0/1911077135 >> v2:172.21.15.177:3300/0 conn(0x7f5ac815cc20 msgr2=0x7f5ac80cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:08.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac67fc700 1 --2- 172.21.15.16:0/1911077135 >> v2:172.21.15.177:3300/0 conn(0x7f5ac815cc20 0x7f5ac80cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.338+0000 7f5ac67fc700 1 -- 172.21.15.16:0/1911077135 >> v1:172.21.15.177:6789/0 conn(0x7f5ac80cf280 legacy=0x7f5ac815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5ac67fc700 1 -- 172.21.15.16:0/1911077135 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac80cbf10 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5ac67fc700 1 --2- 172.21.15.16:0/1911077135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00098e0 tx=0x7f5ab00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=444ea5067d2f540b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5ac5ffb700 1 -- 172.21.15.16:0/1911077135 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000b640 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5ac5ffb700 1 -- 172.21.15.16:0/1911077135 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab000b7a0 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5ac5ffb700 1 -- 172.21.15.16:0/1911077135 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000bb40 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 -- 172.21.15.16:0/1911077135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 msgr2=0x7f5ac80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 --2- 172.21.15.16:0/1911077135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ab00098e0 tx=0x7f5ab00091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 -- 172.21.15.16:0/1911077135 shutdown_connections 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 --2- 172.21.15.16:0/1911077135 >> v2:172.21.15.177:3300/0 conn(0x7f5ac815cc20 0x7f5ac80cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 --2- 172.21.15.16:0/1911077135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cd6a0 0x7f5ac80ced40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 -- 172.21.15.16:0/1911077135 >> 172.21.15.16:0/1911077135 conn(0x7f5ac8099b40 msgr2=0x7f5ac806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 -- 172.21.15.16:0/1911077135 shutdown_connections 2023-10-06T11:42:08.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.339+0000 7f5acd83f700 1 -- 172.21.15.16:0/1911077135 wait complete. 2023-10-06T11:42:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 Processor -- start 2023-10-06T11:42:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 -- start start 2023-10-06T11:42:08.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 0x7f5ac80c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cf280 0x7f5ac815aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac809d6d0 con 0x7f5ac815cc20 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80c9010 con 0x7f5ac80cd6a0 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.341+0000 7f5acd83f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80c92f0 con 0x7f5ac80cf280 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37824/0 (socket says 172.21.15.16:37824) 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 -- 172.21.15.16:0/789932097 learned_addr learned my addr 172.21.15.16:0/789932097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac6ffd700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 0x7f5ac80c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac67fc700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cf280 0x7f5ac815aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 -- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cf280 msgr2=0x7f5ac815aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cf280 0x7f5ac815aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 -- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 msgr2=0x7f5ac80c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 0x7f5ac80c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 -- 172.21.15.16:0/789932097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0009420 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac6ffd700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 0x7f5ac80c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5ac77fe700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000cbc0 tx=0x7f5ac00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000a890 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac000a9f0 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000ad70 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac80c95d0 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.342+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac80c9b90 con 0x7f5ac815cc20 2023-10-06T11:42:08.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.343+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac80c8d80 con 0x7f5ac815cc20 2023-10-06T11:42:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.344+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ac0007300 con 0x7f5ac815cc20 2023-10-06T11:42:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.344+0000 7f5abeffd700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 0x7f5aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.344+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5ac00849c0 con 0x7f5ac815cc20 2023-10-06T11:42:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.344+0000 7f5ac6ffd700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 0x7f5aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.345+0000 7f5ac6ffd700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 0x7f5aa80693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009ce0 tx=0x7f5ab8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.348+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ac0051b50 con 0x7f5ac815cc20 2023-10-06T11:42:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.543+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ac804c380 con 0x7f5ac815cc20 2023-10-06T11:42:08.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.544+0000 7f5abeffd700 1 -- 172.21.15.16:0/789932097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f5ac0055170 con 0x7f5ac815cc20 2023-10-06T11:42:08.546 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:08.547 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 msgr2=0x7f5aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 0x7f5aa80693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009ce0 tx=0x7f5ab8008040 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 msgr2=0x7f5ac8169d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000cbc0 tx=0x7f5ac00019c0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 shutdown_connections 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac815cc20 0x7f5ac8169d20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5aa8066f30 0x7f5aa80693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ac80cd6a0 0x7f5ac80c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 --2- 172.21.15.16:0/789932097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ac80cf280 0x7f5ac815aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 >> 172.21.15.16:0/789932097 conn(0x7f5ac8099b40 msgr2=0x7f5ac80c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 shutdown_connections 2023-10-06T11:42:08.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.546+0000 7f5acd83f700 1 -- 172.21.15.16:0/789932097 wait complete. 2023-10-06T11:42:08.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:08.561 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.682+0000 7f20ebff1700 1 Processor -- start 2023-10-06T11:42:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.682+0000 7f20ebff1700 1 -- start start 2023-10-06T11:42:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ebff1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e4159740 0x7f20e4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ebff1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20e40b4790 0x7f20e40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ebff1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e40b1790 con 0x7f20e40b09d0 2023-10-06T11:42:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ebff1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e40b0e60 con 0x7f20e40b4790 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ebff1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40b1140 con 0x7f20e4159740 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20e40b4790 0x7f20e40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20e40b4790 0x7f20e40b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37832/0 (socket says 172.21.15.16:37832) 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 -- 172.21.15.16:0/3890758071 learned_addr learned my addr 172.21.15.16:0/3890758071 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20e9d8d700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e4159740 0x7f20e4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 -- 172.21.15.16:0/3890758071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e4159740 msgr2=0x7f20e4159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e4159740 0x7f20e4159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 -- 172.21.15.16:0/3890758071 >> v1:172.21.15.177:6789/0 conn(0x7f20e40b09d0 legacy=0x7f20e40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 -- 172.21.15.16:0/3890758071 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40b1420 con 0x7f20e40b4790 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.683+0000 7f20ea58e700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e40b34c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009f20 tx=0x7f20d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236d99d6d367e324 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20e8d8b700 1 -- 172.21.15.16:0/3890758071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d400e070 con 0x7f20e40b4790 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20e8d8b700 1 -- 172.21.15.16:0/3890758071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d400ba60 con 0x7f20e40b4790 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20e8d8b700 1 -- 172.21.15.16:0/3890758071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40092e0 con 0x7f20e40b4790 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 -- 172.21.15.16:0/3890758071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 msgr2=0x7f20e40b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e40b34c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009f20 tx=0x7f20d4009840 comp rx=0 tx=0).stop 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 -- 172.21.15.16:0/3890758071 shutdown_connections 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e40b34c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 --2- 172.21.15.16:0/3890758071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e4159740 0x7f20e4159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 -- 172.21.15.16:0/3890758071 >> 172.21.15.16:0/3890758071 conn(0x7f20e4099a40 msgr2=0x7f20e406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 -- 172.21.15.16:0/3890758071 shutdown_connections 2023-10-06T11:42:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.684+0000 7f20ebff1700 1 -- 172.21.15.16:0/3890758071 wait complete. 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.685+0000 7f20ebff1700 1 Processor -- start 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.685+0000 7f20ebff1700 1 -- start start 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e40b09d0 0x7f20e41698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 0x7f20e409d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e409b010 con 0x7f20e40b4790 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e40c78e0 con 0x7f20e4159740 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ebff1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e409a9c0 con 0x7f20e40b09d0 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37842/0 (socket says 172.21.15.16:37842) 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 -- 172.21.15.16:0/1045397471 learned_addr learned my addr 172.21.15.16:0/1045397471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20ea58e700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 0x7f20e409d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e9d8d700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e40b09d0 0x7f20e41698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 -- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e40b09d0 msgr2=0x7f20e41698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e40b09d0 0x7f20e41698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 -- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 msgr2=0x7f20e409d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 0x7f20e409d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.686+0000 7f20e958c700 1 -- 172.21.15.16:0/1045397471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009ae0 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20ea58e700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 0x7f20e409d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20e958c700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c590 tx=0x7f20d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011590 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8011710 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e409aca0 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8011ad0 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.687+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e409dbb0 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.688+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e404c380 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.688+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f20d801b3b0 con 0x7f20e40b4790 2023-10-06T11:42:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.689+0000 7f20e1ffb700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.689+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f20d8083bf0 con 0x7f20e40b4790 2023-10-06T11:42:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.689+0000 7f20e9d8d700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.690+0000 7f20e9d8d700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 0x7f20d00693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009d10 tx=0x7f20dc0091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:08.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.693+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20d8050c60 con 0x7f20e40b4790 2023-10-06T11:42:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.886+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f20e40b2f40 con 0x7f20e40b4790 2023-10-06T11:42:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.886+0000 7f20e1ffb700 1 -- 172.21.15.16:0/1045397471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f20d8017070 con 0x7f20e40b4790 2023-10-06T11:42:08.889 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 msgr2=0x7f20d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 0x7f20d00693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009d10 tx=0x7f20dc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 msgr2=0x7f20e4158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c590 tx=0x7f20d800c8a0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 shutdown_connections 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f20d0066f30 0x7f20d00693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f20e4159740 0x7f20e409d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f20e40b09d0 0x7f20e41698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 --2- 172.21.15.16:0/1045397471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20e40b4790 0x7f20e4158980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 >> 172.21.15.16:0/1045397471 conn(0x7f20e4099a40 msgr2=0x7f20e415a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 shutdown_connections 2023-10-06T11:42:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:08.888+0000 7f20ebff1700 1 -- 172.21.15.16:0/1045397471 wait complete. 2023-10-06T11:42:08.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.022+0000 7fd52089a700 1 Processor -- start 2023-10-06T11:42:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.022+0000 7fd52089a700 1 -- start start 2023-10-06T11:42:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd52089a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd51c157580 0x7fd51c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd52089a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd51c0b2bd0 0x7fd51c0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd52089a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c0ce2e0 con 0x7fd51c06aee0 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd52089a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0ca170 con 0x7fd51c157580 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd52089a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c0cd6d0 con 0x7fd51c0b2bd0 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd51c157580 0x7fd51c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd519d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd51c06aee0 0x7fd51c0b24c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35878/0 (socket says 172.21.15.16:35878) 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd519d9b700 1 -- 172.21.15.16:0/2416795839 learned_addr learned my addr 172.21.15.16:0/2416795839 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51ad9d700 1 --2- 172.21.15.16:0/2416795839 >> v2:172.21.15.177:3300/0 conn(0x7fd51c0b2bd0 0x7fd51c0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 -- 172.21.15.16:0/2416795839 >> v2:172.21.15.177:3300/0 conn(0x7fd51c0b2bd0 msgr2=0x7fd51c0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 --2- 172.21.15.16:0/2416795839 >> v2:172.21.15.177:3300/0 conn(0x7fd51c0b2bd0 0x7fd51c0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 -- 172.21.15.16:0/2416795839 >> v1:172.21.15.16:6789/0 conn(0x7fd51c06aee0 legacy=0x7fd51c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 -- 172.21.15.16:0/2416795839 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0cd9b0 con 0x7fd51c157580 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51959a700 1 -- 172.21.15.16:0/2416795839 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861714721 0 0) 0x7fd51c0ce2e0 con 0x7fd51c06aee0 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51a59c700 1 --2- 172.21.15.16:0/2416795839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c157580 0x7fd51c0c9c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0ce250 tx=0x7fd50c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1a117a0a7d2aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51959a700 1 -- 172.21.15.16:0/2416795839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c00e070 con 0x7fd51c157580 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51959a700 1 -- 172.21.15.16:0/2416795839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c00bc30 con 0x7fd51c157580 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.023+0000 7fd51959a700 1 -- 172.21.15.16:0/2416795839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c012430 con 0x7fd51c157580 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 -- 172.21.15.16:0/2416795839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c157580 msgr2=0x7fd51c0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 --2- 172.21.15.16:0/2416795839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c157580 0x7fd51c0c9c30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0ce250 tx=0x7fd50c009780 comp rx=0 tx=0).stop 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 -- 172.21.15.16:0/2416795839 shutdown_connections 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 --2- 172.21.15.16:0/2416795839 >> v2:172.21.15.177:3300/0 conn(0x7fd51c0b2bd0 0x7fd51c0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 --2- 172.21.15.16:0/2416795839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c157580 0x7fd51c0c9c30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 -- 172.21.15.16:0/2416795839 >> 172.21.15.16:0/2416795839 conn(0x7fd51c068030 msgr2=0x7fd51c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 -- 172.21.15.16:0/2416795839 shutdown_connections 2023-10-06T11:42:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.024+0000 7fd52089a700 1 -- 172.21.15.16:0/2416795839 wait complete. 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.025+0000 7fd52089a700 1 Processor -- start 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.025+0000 7fd52089a700 1 -- start start 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd51c0b2bd0 0x7fd51c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51c157580 0x7fd51c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c159bf0 con 0x7fd51c06aee0 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c15a490 con 0x7fd51c0b2bd0 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c15c4a0 con 0x7fd51c157580 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37862/0 (socket says 172.21.15.16:37862) 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 -- 172.21.15.16:0/1163393125 learned_addr learned my addr 172.21.15.16:0/1163393125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd519d9b700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd51c0b2bd0 0x7fd51c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51ad9d700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51c157580 0x7fd51c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 -- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51c157580 msgr2=0x7fd51c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51c157580 0x7fd51c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 -- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd51c0b2bd0 msgr2=0x7fd51c1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd51c0b2bd0 0x7fd51c1524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 -- 172.21.15.16:0/1163393125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c009a20 con 0x7fd51c06aee0 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd51a59c700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd50c013000 tx=0x7fd50c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c012d50 con 0x7fd51c06aee0 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c027470 con 0x7fd51c06aee0 2023-10-06T11:42:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c159e40 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c027830 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.027+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c153790 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.028+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c154700 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.028+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd50c024070 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.028+0000 7fd5127fc700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 0x7fd4fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.028+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd50c086b90 con 0x7fd51c06aee0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.029+0000 7fd519d9b700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 0x7fd4fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.029+0000 7fd519d9b700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 0x7fd4fc0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd51c15b970 tx=0x7fd504007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.032+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd50c053ca0 con 0x7fd51c06aee0 2023-10-06T11:42:09.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.226+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd51c070e40 con 0x7fd51c06aee0 2023-10-06T11:42:09.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.227+0000 7fd5127fc700 1 -- 172.21.15.16:0/1163393125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd50c02c020 con 0x7fd51c06aee0 2023-10-06T11:42:09.229 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 msgr2=0x7fd4fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 0x7fd4fc0693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd51c15b970 tx=0x7fd504007d30 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 msgr2=0x7fd51c15a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd50c013000 tx=0x7fd50c019000 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 shutdown_connections 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd51c06aee0 0x7fd51c15a920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4fc066f30 0x7fd4fc0693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd51c157580 0x7fd51c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 --2- 172.21.15.16:0/1163393125 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd51c0b2bd0 0x7fd51c1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 >> 172.21.15.16:0/1163393125 conn(0x7fd51c068030 msgr2=0x7fd51c0ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 shutdown_connections 2023-10-06T11:42:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.230+0000 7fd52089a700 1 -- 172.21.15.16:0/1163393125 wait complete. 2023-10-06T11:42:09.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.364+0000 7fd21cdaf700 1 Processor -- start 2023-10-06T11:42:09.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.364+0000 7fd21cdaf700 1 -- start start 2023-10-06T11:42:09.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 0x7fd2180b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd218159b10 0x7fd2180cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180cead0 con 0x7fd218159740 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180affc0 con 0x7fd218159b10 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180cdec0 con 0x7fd2180cda10 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd218159b10 0x7fd2180cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd218159740 0x7fd2180b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35884/0 (socket says 172.21.15.16:35884) 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21659c700 1 -- 172.21.15.16:0/405592302 learned_addr learned my addr 172.21.15.16:0/405592302 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd215d9b700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 0x7fd2180b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 -- 172.21.15.16:0/405592302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 msgr2=0x7fd2180b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 0x7fd2180b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 -- 172.21.15.16:0/405592302 >> v1:172.21.15.16:6789/0 conn(0x7fd218159740 legacy=0x7fd2180b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 -- 172.21.15.16:0/405592302 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180ce1a0 con 0x7fd218159b10 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd215d9b700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 0x7fd2180b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21559a700 1 -- 172.21.15.16:0/405592302 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179985164 0 0) 0x7fd2180cead0 con 0x7fd218159740 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd216d9d700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd2180cef60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2080098e0 tx=0x7fd2080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82c1fc431053459 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21559a700 1 -- 172.21.15.16:0/405592302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208009e80 con 0x7fd218159b10 2023-10-06T11:42:09.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21559a700 1 -- 172.21.15.16:0/405592302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20800b860 con 0x7fd218159b10 2023-10-06T11:42:09.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21559a700 1 -- 172.21.15.16:0/405592302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800bbe0 con 0x7fd218159b10 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- 172.21.15.16:0/405592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 msgr2=0x7fd2180cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd2180cef60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2080098e0 tx=0x7fd2080091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- 172.21.15.16:0/405592302 shutdown_connections 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd2180cef60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/405592302 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2180cda10 0x7fd2180b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- 172.21.15.16:0/405592302 >> 172.21.15.16:0/405592302 conn(0x7fd218099a40 msgr2=0x7fd21806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- 172.21.15.16:0/405592302 shutdown_connections 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.365+0000 7fd21cdaf700 1 -- 172.21.15.16:0/405592302 wait complete. 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.366+0000 7fd21cdaf700 1 Processor -- start 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.366+0000 7fd21cdaf700 1 -- start start 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 0x7fd21816cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd218159740 0x7fd2180b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180b0230 con 0x7fd218159b10 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180b0510 con 0x7fd2180cda10 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21cdaf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180b07f0 con 0x7fd218159740 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37884/0 (socket says 172.21.15.16:37884) 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 -- 172.21.15.16:0/3692415602 learned_addr learned my addr 172.21.15.16:0/3692415602 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd215d9b700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd218159740 0x7fd2180b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21659c700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 0x7fd21816cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 -- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd218159740 msgr2=0x7fd2180b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd218159740 0x7fd2180b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 -- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 msgr2=0x7fd21816cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 0x7fd21816cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd216d9d700 1 -- 172.21.15.16:0/3692415602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009420 con 0x7fd218159b10 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.367+0000 7fd21659c700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 0x7fd21816cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd216d9d700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd2080091e0 tx=0x7fd208017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20800bbe0 con 0x7fd218159b10 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208011420 con 0x7fd218159b10 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080117e0 con 0x7fd218159b10 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2180b0ad0 con 0x7fd218159b10 2023-10-06T11:42:09.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.368+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21816d2f0 con 0x7fd218159b10 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.369+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2180b0ff0 con 0x7fd218159b10 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.369+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd208011c10 con 0x7fd218159b10 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.370+0000 7fd2067fc700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 0x7fd1f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.370+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd208085cc0 con 0x7fd218159b10 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.370+0000 7fd21659c700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 0x7fd1f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.370+0000 7fd21659c700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 0x7fd1f80693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd210001560 tx=0x7fd210009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.375+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd208052f00 con 0x7fd218159b10 2023-10-06T11:42:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.564+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd21804c380 con 0x7fd218159b10 2023-10-06T11:42:09.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.565+0000 7fd2067fc700 1 -- 172.21.15.16:0/3692415602 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd208015020 con 0x7fd218159b10 2023-10-06T11:42:09.567 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 msgr2=0x7fd1f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 0x7fd1f80693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd210001560 tx=0x7fd210009040 comp rx=0 tx=0).stop 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 msgr2=0x7fd218159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd2080091e0 tx=0x7fd208017000 comp rx=0 tx=0).stop 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 shutdown_connections 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd218159b10 0x7fd218159f10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd1f8066f30 0x7fd1f80693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2180cda10 0x7fd21816cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 --2- 172.21.15.16:0/3692415602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd218159740 0x7fd2180b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.566+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 >> 172.21.15.16:0/3692415602 conn(0x7fd218099a40 msgr2=0x7fd21806dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.567+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 shutdown_connections 2023-10-06T11:42:09.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.567+0000 7fd21cdaf700 1 -- 172.21.15.16:0/3692415602 wait complete. 2023-10-06T11:42:09.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.701+0000 7fd45144d700 1 Processor -- start 2023-10-06T11:42:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.701+0000 7fd45144d700 1 -- start start 2023-10-06T11:42:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd45144d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd45144d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd44c0cb660 0x7fd44c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd45144d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c13c9f0 con 0x7fd44c15d500 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd45144d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c15e270 con 0x7fd44c0cb660 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd45144d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15e050 con 0x7fd44c0cb290 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd44c0cb660 0x7fd44c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd44c15d500 0x7fd44c15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35894/0 (socket says 172.21.15.16:35894) 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd44c0cb660 0x7fd44c0cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37900/0 (socket says 172.21.15.16:37900) 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44b7fe700 1 -- 172.21.15.16:0/2228561738 learned_addr learned my addr 172.21.15.16:0/2228561738 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44affd700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 -- 172.21.15.16:0/2228561738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 msgr2=0x7fd44c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd449ffb700 1 -- 172.21.15.16:0/2228561738 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386957927 0 0) 0x7fd43c001110 con 0x7fd44c15d500 2023-10-06T11:42:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 -- 172.21.15.16:0/2228561738 >> v1:172.21.15.16:6789/0 conn(0x7fd44c15d500 legacy=0x7fd44c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 -- 172.21.15.16:0/2228561738 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c06dff0 con 0x7fd44c0cb660 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.702+0000 7fd44a7fc700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c0cc600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098e0 tx=0x7fd4340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb20366013e7c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd449ffb700 1 -- 172.21.15.16:0/2228561738 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400b640 con 0x7fd44c0cb660 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd449ffb700 1 -- 172.21.15.16:0/2228561738 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43400b7a0 con 0x7fd44c0cb660 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd449ffb700 1 -- 172.21.15.16:0/2228561738 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400bb40 con 0x7fd44c0cb660 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 -- 172.21.15.16:0/2228561738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 msgr2=0x7fd44c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c0cc600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098e0 tx=0x7fd4340091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 -- 172.21.15.16:0/2228561738 shutdown_connections 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 --2- 172.21.15.16:0/2228561738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c0cc600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 -- 172.21.15.16:0/2228561738 >> 172.21.15.16:0/2228561738 conn(0x7fd44c067c90 msgr2=0x7fd44c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 -- 172.21.15.16:0/2228561738 shutdown_connections 2023-10-06T11:42:09.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.703+0000 7fd45144d700 1 -- 172.21.15.16:0/2228561738 wait complete. 2023-10-06T11:42:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 Processor -- start 2023-10-06T11:42:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 -- start start 2023-10-06T11:42:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd44c15d500 0x7fd44c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c16ac10 con 0x7fd44c0cb660 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c16b2b0 con 0x7fd44c15d500 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd45144d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c16a650 con 0x7fd44c0cb290 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd44a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd44a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37908/0 (socket says 172.21.15.16:37908) 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd44a7fc700 1 -- 172.21.15.16:0/1634996734 learned_addr learned my addr 172.21.15.16:0/1634996734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.705+0000 7fd44affd700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44b7fe700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd44c15d500 0x7fd44c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 -- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 msgr2=0x7fd44c13be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c13be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 -- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd44c15d500 msgr2=0x7fd44c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd44c15d500 0x7fd44c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 -- 172.21.15.16:0/1634996734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd434009420 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44affd700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd44a7fc700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd434013000 tx=0x7fd434017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43400bb20 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434011420 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4340117a0 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c16a930 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.706+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0c45b0 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.707+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c16ada0 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.707+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd434011b10 con 0x7fd44c0cb660 2023-10-06T11:42:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.708+0000 7fd442ffd700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 0x7fd42c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.708+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd4340859e0 con 0x7fd44c0cb660 2023-10-06T11:42:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.708+0000 7fd44affd700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 0x7fd42c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.708+0000 7fd44affd700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 0x7fd42c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd44c15acc0 tx=0x7fd44400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:09.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.713+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd44c16ada0 con 0x7fd44c0cb660 2023-10-06T11:42:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.906+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd44c16ada0 con 0x7fd44c0cb660 2023-10-06T11:42:09.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.906+0000 7fd442ffd700 1 -- 172.21.15.16:0/1634996734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd434015020 con 0x7fd44c0cb660 2023-10-06T11:42:09.907 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 msgr2=0x7fd42c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 0x7fd42c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd44c15acc0 tx=0x7fd44400b040 comp rx=0 tx=0).stop 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 msgr2=0x7fd44c169b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd434013000 tx=0x7fd434017000 comp rx=0 tx=0).stop 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 shutdown_connections 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd42c066f30 0x7fd42c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd44c15d500 0x7fd44c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd44c0cb290 0x7fd44c13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 --2- 172.21.15.16:0/1634996734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd44c0cb660 0x7fd44c169b90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 >> 172.21.15.16:0/1634996734 conn(0x7fd44c067c90 msgr2=0x7fd44c0c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 shutdown_connections 2023-10-06T11:42:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:09.908+0000 7fd45144d700 1 -- 172.21.15.16:0/1634996734 wait complete. 2023-10-06T11:42:09.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.039+0000 7f1beb20e700 1 Processor -- start 2023-10-06T11:42:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.039+0000 7f1beb20e700 1 -- start start 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1beb20e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 0x7f1be40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1beb20e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be40cdb10 con 0x7f1be40cf450 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1beb20e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be40cf8d0 con 0x7f1be40cd6a0 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1beb20e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be40b3110 con 0x7f1be415cc20 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1be97ab700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1be40cf450 0x7f1be40c5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35906/0 (socket says 172.21.15.16:35906) 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1be97ab700 1 -- 172.21.15.16:0/3556027192 learned_addr learned my addr 172.21.15.16:0/3556027192 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.040+0000 7f1be8faa700 1 --2- 172.21.15.16:0/3556027192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 0x7f1be40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118329809 0 0) 0x7f1be40cdb10 con 0x7f1be40cf450 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd4003570 con 0x7f1be40cf450 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86171930 0 0) 0x7f1bd0001110 con 0x7f1be40cd6a0 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be40cdb10 con 0x7f1be40cd6a0 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291540905 0 0) 0x7f1bd4003570 con 0x7f1be40cf450 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be8faa700 1 -- 172.21.15.16:0/3556027192 >> v1:172.21.15.177:6789/0 conn(0x7f1be40cd6a0 legacy=0x7f1be40ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be8faa700 1 -- 172.21.15.16:0/3556027192 >> v1:172.21.15.16:6789/0 conn(0x7f1be40cf450 legacy=0x7f1be40c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be8faa700 1 -- 172.21.15.16:0/3556027192 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be40ccb20 con 0x7f1be415cc20 2023-10-06T11:42:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be8faa700 1 --2- 172.21.15.16:0/3556027192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 0x7f1be40cd160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009910 tx=0x7f1bd80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571e423daaa63ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc0017f0 con 0x7f1be415cc20 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8011420 con 0x7f1be415cc20 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.041+0000 7f1be37fe700 1 -- 172.21.15.16:0/3556027192 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80117c0 con 0x7f1be415cc20 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 -- 172.21.15.16:0/3556027192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 msgr2=0x7f1be40cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 --2- 172.21.15.16:0/3556027192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 0x7f1be40cd160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009910 tx=0x7f1bd80091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 -- 172.21.15.16:0/3556027192 shutdown_connections 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 --2- 172.21.15.16:0/3556027192 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be415cc20 0x7f1be40cd160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 -- 172.21.15.16:0/3556027192 >> 172.21.15.16:0/3556027192 conn(0x7f1be4099b40 msgr2=0x7f1be406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 -- 172.21.15.16:0/3556027192 shutdown_connections 2023-10-06T11:42:10.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.042+0000 7f1beb20e700 1 -- 172.21.15.16:0/3556027192 wait complete. 2023-10-06T11:42:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.043+0000 7f1beb20e700 1 Processor -- start 2023-10-06T11:42:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.043+0000 7f1beb20e700 1 -- start start 2023-10-06T11:42:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 0x7f1be409b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 0x7f1be414c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be414c800 con 0x7f1be415cc20 2023-10-06T11:42:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be409aeb0 con 0x7f1be40cd6a0 2023-10-06T11:42:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1beb20e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be409a2a0 con 0x7f1be40cf450 2023-10-06T11:42:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37922/0 (socket says 172.21.15.16:37922) 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 -- 172.21.15.16:0/1838722809 learned_addr learned my addr 172.21.15.16:0/1838722809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be8faa700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 0x7f1be409b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be3fff700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 0x7f1be414c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 -- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 msgr2=0x7f1be414c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 0x7f1be414c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 -- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 msgr2=0x7f1be409b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 0x7f1be409b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 -- 172.21.15.16:0/1838722809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd8009420 con 0x7f1be415cc20 2023-10-06T11:42:10.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be3fff700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 0x7f1be414c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be8faa700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 0x7f1be409b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.044+0000 7f1be97ab700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001560 tx=0x7f1bdc007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.045+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00ce10 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.045+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bdc004dc0 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.045+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be409a580 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.045+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc00ab50 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.045+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be409ab40 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.046+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be404c380 con 0x7f1be415cc20 2023-10-06T11:42:10.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.046+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1bdc007150 con 0x7f1be415cc20 2023-10-06T11:42:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.046+0000 7f1be0ff9700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 0x7f1bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.047+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1bdc0843a0 con 0x7f1be415cc20 2023-10-06T11:42:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.047+0000 7f1be8faa700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 0x7f1bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.047+0000 7f1be8faa700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 0x7f1bc80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80030c0 tx=0x7f1bd8016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.051+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bdc051410 con 0x7f1be415cc20 2023-10-06T11:42:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.289+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1be40cbf10 con 0x7f1be415cc20 2023-10-06T11:42:10.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.289+0000 7f1be0ff9700 1 -- 172.21.15.16:0/1838722809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f1bdc018090 con 0x7f1be415cc20 2023-10-06T11:42:10.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:10.293 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 msgr2=0x7f1bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 0x7f1bc80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bd80030c0 tx=0x7f1bd8016000 comp rx=0 tx=0).stop 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 msgr2=0x7f1be40c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001560 tx=0x7f1bdc007d20 comp rx=0 tx=0).stop 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 shutdown_connections 2023-10-06T11:42:10.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be415cc20 0x7f1be40c9f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1bc8066f30 0x7f1bc80693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1be40cd6a0 0x7f1be409b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 --2- 172.21.15.16:0/1838722809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1be40cf450 0x7f1be414c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 >> 172.21.15.16:0/1838722809 conn(0x7f1be4099b40 msgr2=0x7f1be415b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 shutdown_connections 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.291+0000 7f1beb20e700 1 -- 172.21.15.16:0/1838722809 wait complete. 2023-10-06T11:42:10.295 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:10.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:10.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.422+0000 7f0edacc1700 1 Processor -- start 2023-10-06T11:42:10.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 -- start start 2023-10-06T11:42:10.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ed415cc20 0x7f0ed40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ed40cd6a0 0x7f0ed40ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed40cdb10 con 0x7f0ed40cf280 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed40b3110 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0edacc1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed40ccb20 con 0x7f0ed415cc20 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.423+0000 7f0ed3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ed40cd6a0 0x7f0ed40ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ed40cd6a0 0x7f0ed40ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37926/0 (socket says 172.21.15.16:37926) 2023-10-06T11:42:10.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 -- 172.21.15.16:0/4052978380 learned_addr learned my addr 172.21.15.16:0/4052978380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed8a5d700 1 --2- 172.21.15.16:0/4052978380 >> v2:172.21.15.177:3300/0 conn(0x7f0ed415cc20 0x7f0ed40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 -- 172.21.15.16:0/4052978380 >> v2:172.21.15.177:3300/0 conn(0x7f0ed415cc20 msgr2=0x7f0ed40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 --2- 172.21.15.16:0/4052978380 >> v2:172.21.15.177:3300/0 conn(0x7f0ed415cc20 0x7f0ed40cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 -- 172.21.15.16:0/4052978380 >> v1:172.21.15.177:6789/0 conn(0x7f0ed40cf280 legacy=0x7f0ed415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 -- 172.21.15.16:0/4052978380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed40cbf10 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed3fff700 1 --2- 172.21.15.16:0/4052978380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed40cd6a0 0x7f0ed40ced40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4089130 tx=0x7f0ec4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610197fb5b3d0b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed37fe700 1 -- 172.21.15.16:0/4052978380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400e070 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed37fe700 1 -- 172.21.15.16:0/4052978380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec400ba10 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0ed37fe700 1 -- 172.21.15.16:0/4052978380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40092e0 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.424+0000 7f0edacc1700 1 -- 172.21.15.16:0/4052978380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed40cd6a0 msgr2=0x7f0ed40ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 --2- 172.21.15.16:0/4052978380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed40cd6a0 0x7f0ed40ced40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4089130 tx=0x7f0ec4009840 comp rx=0 tx=0).stop 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 -- 172.21.15.16:0/4052978380 shutdown_connections 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 --2- 172.21.15.16:0/4052978380 >> v2:172.21.15.177:3300/0 conn(0x7f0ed415cc20 0x7f0ed40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 --2- 172.21.15.16:0/4052978380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed40cd6a0 0x7f0ed40ced40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 -- 172.21.15.16:0/4052978380 >> 172.21.15.16:0/4052978380 conn(0x7f0ed4099b40 msgr2=0x7f0ed406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 -- 172.21.15.16:0/4052978380 shutdown_connections 2023-10-06T11:42:10.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.425+0000 7f0edacc1700 1 -- 172.21.15.16:0/4052978380 wait complete. 2023-10-06T11:42:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.426+0000 7f0edacc1700 1 Processor -- start 2023-10-06T11:42:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.426+0000 7f0edacc1700 1 -- start start 2023-10-06T11:42:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 0x7f0ed40b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 0x7f0ed40b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40b6370 con 0x7f0ed415cc20 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed416dd90 con 0x7f0ed40cf280 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed416e070 con 0x7f0ed40cd6a0 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37932/0 (socket says 172.21.15.16:37932) 2023-10-06T11:42:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 -- 172.21.15.16:0/3488718698 learned_addr learned my addr 172.21.15.16:0/3488718698 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed3fff700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 0x7f0ed40b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed8a5d700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 0x7f0ed40b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 -- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 msgr2=0x7f0ed40b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 0x7f0ed40b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 -- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 msgr2=0x7f0ed40b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 0x7f0ed40b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 -- 172.21.15.16:0/3488718698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4009ae0 con 0x7f0ed415cc20 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed3fff700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 0x7f0ed40b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed8a5d700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 0x7f0ed40b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed925e700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b4e0 tx=0x7f0ebc009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00f5e0 con 0x7f0ed415cc20 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc00f760 con 0x7f0ed415cc20 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed416e350 con 0x7f0ed415cc20 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00fb20 con 0x7f0ed415cc20 2023-10-06T11:42:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.427+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed40b4a10 con 0x7f0ed415cc20 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.428+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed404c380 con 0x7f0ed415cc20 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.429+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ebc01a400 con 0x7f0ed415cc20 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.429+0000 7f0ed0ff9700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 0x7f0eb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.429+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0ebc084200 con 0x7f0ed415cc20 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.429+0000 7f0ed8a5d700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 0x7f0eb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.430+0000 7f0ed8a5d700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 0x7f0eb40693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc008a80 tx=0x7f0ecc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.433+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ebc051390 con 0x7f0ed415cc20 2023-10-06T11:42:10.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.629+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed40cc1f0 con 0x7f0ed415cc20 2023-10-06T11:42:10.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.629+0000 7f0ed0ff9700 1 -- 172.21.15.16:0/3488718698 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f0ebc00fed0 con 0x7f0ed415cc20 2023-10-06T11:42:10.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:10.631 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 msgr2=0x7f0eb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 0x7f0eb40693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc008a80 tx=0x7f0ecc008040 comp rx=0 tx=0).stop 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 msgr2=0x7f0ed40c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00b4e0 tx=0x7f0ebc009060 comp rx=0 tx=0).stop 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 shutdown_connections 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ed415cc20 0x7f0ed40c6e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0eb4066f30 0x7f0eb40693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ed40cd6a0 0x7f0ed40b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 --2- 172.21.15.16:0/3488718698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ed40cf280 0x7f0ed40b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 >> 172.21.15.16:0/3488718698 conn(0x7f0ed4099b40 msgr2=0x7f0ed415cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 shutdown_connections 2023-10-06T11:42:10.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.631+0000 7f0edacc1700 1 -- 172.21.15.16:0/3488718698 wait complete. 2023-10-06T11:42:10.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:10.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.762+0000 7f8e8139b700 1 Processor -- start 2023-10-06T11:42:10.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.762+0000 7f8e8139b700 1 -- start start 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e8139b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 0x7f8e7c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e8139b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8e7c0cf380 0x7f8e7c0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e8139b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7c0cbf20 con 0x7f8e7c159980 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e8139b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7c0cc200 con 0x7f8e7c0cf380 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e8139b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c0cc4e0 con 0x7f8e7c1595b0 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e7b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8e7c0cf380 0x7f8e7c0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e7affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 0x7f8e7c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.763+0000 7f8e7a7fc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8e7c159980 0x7f8e7c0cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:40184/0 (socket says 172.21.15.16:40184) 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7a7fc700 1 -- 172.21.15.16:0/256518744 learned_addr learned my addr 172.21.15.16:0/256518744 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7b7fe700 1 -- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 msgr2=0x7f8e7c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7b7fe700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 0x7f8e7c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7b7fe700 1 -- 172.21.15.16:0/256518744 >> v1:172.21.15.177:6789/0 conn(0x7f8e7c159980 legacy=0x7f8e7c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7b7fe700 1 -- 172.21.15.16:0/256518744 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c0cc7c0 con 0x7f8e7c0cf380 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7affd700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 0x7f8e7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e7b7fe700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c0cd680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e60 tx=0x7f8e74009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f33945909dc5bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e79ffb700 1 -- 172.21.15.16:0/256518744 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400e070 con 0x7f8e7c0cf380 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e79ffb700 1 -- 172.21.15.16:0/256518744 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7400bb60 con 0x7f8e7c0cf380 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.764+0000 7f8e79ffb700 1 -- 172.21.15.16:0/256518744 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e74012430 con 0x7f8e7c0cf380 2023-10-06T11:42:10.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 -- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 msgr2=0x7f8e7c0cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c0cd680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009e60 tx=0x7f8e74009780 comp rx=0 tx=0).stop 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 -- 172.21.15.16:0/256518744 shutdown_connections 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c0cd680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 --2- 172.21.15.16:0/256518744 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c1595b0 0x7f8e7c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 -- 172.21.15.16:0/256518744 >> 172.21.15.16:0/256518744 conn(0x7f8e7c099b40 msgr2=0x7f8e7c087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 -- 172.21.15.16:0/256518744 shutdown_connections 2023-10-06T11:42:10.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.765+0000 7f8e8139b700 1 -- 172.21.15.16:0/256518744 wait complete. 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.766+0000 7f8e8139b700 1 Processor -- start 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.766+0000 7f8e8139b700 1 -- start start 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c159980 0x7f8e7c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c0c9a20 con 0x7f8e7c1595b0 2023-10-06T11:42:10.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c0c8bb0 con 0x7f8e7c0cf380 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e8139b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1554d0 con 0x7f8e7c159980 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37948/0 (socket says 172.21.15.16:37948) 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 -- 172.21.15.16:0/3550073993 learned_addr learned my addr 172.21.15.16:0/3550073993 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7affd700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7b7fe700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c159980 0x7f8e7c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 -- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c159980 msgr2=0x7f8e7c154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c159980 0x7f8e7c154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 -- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 msgr2=0x7f8e7c15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 -- 172.21.15.16:0/3550073993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c009420 con 0x7f8e7c1595b0 2023-10-06T11:42:10.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7affd700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.767+0000 7f8e7a7fc700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c590 tx=0x7f8e64001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64005b30 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e64005cb0 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e74009a20 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64007320 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c0c90f0 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.768+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c0c9b90 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.769+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8e6401b3b0 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.769+0000 7f8e72ffd700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 0x7f8e5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.769+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8e64083b70 con 0x7f8e7c1595b0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.769+0000 7f8e7affd700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 0x7f8e5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:10.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.770+0000 7f8e7affd700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 0x7f8e5c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0093f0 tx=0x7f8e6c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:10.774+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e64050c80 con 0x7f8e7c1595b0 2023-10-06T11:42:11.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.010+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e7c04c380 con 0x7f8e7c1595b0 2023-10-06T11:42:11.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.011+0000 7f8e72ffd700 1 -- 172.21.15.16:0/3550073993 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f8e64017070 con 0x7f8e7c1595b0 2023-10-06T11:42:11.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:11.015 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:11.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 msgr2=0x7f8e5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 0x7f8e5c0693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c0093f0 tx=0x7f8e6c008040 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 msgr2=0x7f8e7c0c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400c590 tx=0x7f8e64001020 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 shutdown_connections 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8e5c066f30 0x7f8e5c0693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8e7c159980 0x7f8e7c154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e7c0cf380 0x7f8e7c15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 --2- 172.21.15.16:0/3550073993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e7c1595b0 0x7f8e7c0c8670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 >> 172.21.15.16:0/3550073993 conn(0x7f8e7c099b40 msgr2=0x7f8e7c0c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 shutdown_connections 2023-10-06T11:42:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.013+0000 7f8e8139b700 1 -- 172.21.15.16:0/3550073993 wait complete. 2023-10-06T11:42:11.017 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:11.428 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:41:55.893171+0000', 'modified': '2023-10-06T11:41:59.529047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6837/1602610299', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6835', 'nonce': 1602610299}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 1602610299}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:42:11.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:42:11.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T11:42:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.553+0000 7f45cfaf6700 1 Processor -- start 2023-10-06T11:42:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45cfaf6700 1 -- start start 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45cfaf6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f45c80ce1e0 0x7f45c806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45cfaf6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c80c88e0 con 0x7f45c8159240 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45cfaf6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c80c8d70 con 0x7f45c806c5d0 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45cfaf6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c80c7cd0 con 0x7f45c80ce1e0 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45ce093700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f45c8159240 0x7f45c8159680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35922/0 (socket says 172.21.15.16:35922) 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.554+0000 7f45ce093700 1 -- 172.21.15.16:0/126480826 learned_addr learned my addr 172.21.15.16:0/126480826 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cd091700 1 --2- 172.21.15.16:0/126480826 >> v2:172.21.15.177:3300/0 conn(0x7f45c80ce1e0 0x7f45c806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891753070 0 0) 0x7f45c80c88e0 con 0x7f45c8159240 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b0003570 con 0x7f45c8159240 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557093640 0 0) 0x7f45c80c8d70 con 0x7f45c806c5d0 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c80c88e0 con 0x7f45c806c5d0 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97609111 0 0) 0x7f45b0003570 con 0x7f45c8159240 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 >> v2:172.21.15.177:3300/0 conn(0x7f45c80ce1e0 msgr2=0x7f45c806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 --2- 172.21.15.16:0/126480826 >> v2:172.21.15.177:3300/0 conn(0x7f45c80ce1e0 0x7f45c806c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 >> v1:172.21.15.177:6789/0 conn(0x7f45c806c5d0 legacy=0x7f45c80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80c7fb0 con 0x7f45c8159240 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f45b8001470 con 0x7f45c8159240 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 >> v1:172.21.15.16:6789/0 conn(0x7f45c8159240 legacy=0x7f45c8159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45b0003c20 0x7f45b0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b0004810 0x7f45b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 0x7f45b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c80c7cd0 con 0x7f45b00051c0 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b000c130 con 0x7f45b0003c20 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b000c2f0 con 0x7f45b0004810 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cd091700 1 --2- 172.21.15.16:0/126480826 >> v2:172.21.15.177:3300/0 conn(0x7f45c80ce1e0 0x7f45c806c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45ce093700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 0x7f45b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cd892700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45b0003c20 0x7f45b0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.555+0000 7f45cd091700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b0004810 0x7f45b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 -- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b0004810 msgr2=0x7f45b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b0004810 0x7f45b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 -- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45b0003c20 msgr2=0x7f45b0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45b0003c20 0x7f45b0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 -- 172.21.15.16:0/126480826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0003570 con 0x7f45b00051c0 2023-10-06T11:42:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 -- 172.21.15.16:0/126480826 reap_dead start 2023-10-06T11:42:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45ce093700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 0x7f45b000ba60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c2d0 tx=0x7f45b8001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2454715fb6a9901 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b800cba0 con 0x7f45b00051c0 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b800cd20 con 0x7f45b00051c0 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cc890700 1 -- 172.21.15.16:0/126480826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8005b70 con 0x7f45b00051c0 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 -- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 msgr2=0x7f45b000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 0x7f45b000ba60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c2d0 tx=0x7f45b8001e10 comp rx=0 tx=0).stop 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 -- 172.21.15.16:0/126480826 shutdown_connections 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/126480826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b00051c0 0x7f45b000ba60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 -- 172.21.15.16:0/126480826 >> 172.21.15.16:0/126480826 conn(0x7f45c806d520 msgr2=0x7f45c806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 -- 172.21.15.16:0/126480826 shutdown_connections 2023-10-06T11:42:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.556+0000 7f45cfaf6700 1 -- 172.21.15.16:0/126480826 wait complete. 2023-10-06T11:42:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.557+0000 7f45cfaf6700 1 Processor -- start 2023-10-06T11:42:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 -- start start 2023-10-06T11:42:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 0x7f45c80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 0x7f45c80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c814a610 con 0x7f45c8149c60 2023-10-06T11:42:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c80acd10 con 0x7f45c80b0290 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cfaf6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c80c5240 con 0x7f45b00051c0 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37978/0 (socket says 172.21.15.16:37978) 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cd091700 1 -- 172.21.15.16:0/1128757388 learned_addr learned my addr 172.21.15.16:0/1128757388 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45cd892700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 0x7f45c80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.558+0000 7f45ce093700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 0x7f45c80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 -- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 msgr2=0x7f45c80b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 0x7f45c80b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 -- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 msgr2=0x7f45c80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 0x7f45c80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 -- 172.21.15.16:0/1128757388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80b0fc0 con 0x7f45c8149c60 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd892700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 0x7f45c80b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45ce093700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 0x7f45c80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cd091700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45bc011960 tx=0x7f45bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45c5ffb700 1 -- 172.21.15.16:0/1128757388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc011ea0 con 0x7f45c8149c60 2023-10-06T11:42:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45c5ffb700 1 -- 172.21.15.16:0/1128757388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc00f8e0 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c80b12a0 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45c5ffb700 1 -- 172.21.15.16:0/1128757388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00fc60 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.559+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c80acf60 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.561+0000 7f45c5ffb700 1 -- 172.21.15.16:0/1128757388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45bc015460 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.561+0000 7f45c5ffb700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 0x7f45b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.561+0000 7f45c5ffb700 1 -- 172.21.15.16:0/1128757388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45bc084ff0 con 0x7f45c8149c60 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.561+0000 7f45cd892700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 0x7f45b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.562+0000 7f45cd892700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 0x7f45b40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45c00042e0 tx=0x7f45c0001c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.563+0000 7f45d8deeb80 1 Processor -- start 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.563+0000 7f45d8deeb80 1 -- start start 2023-10-06T11:42:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f654ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f5d4c80 con 0x56541f6579a0 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f5aba70 con 0x56541f655bd0 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541f5a9270 con 0x56541f69ba90 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37980/0 (socket says 172.21.15.16:37980) 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 -- 172.21.15.16:0/3400117433 learned_addr learned my addr 172.21.15.16:0/3400117433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd892700 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45ce093700 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f654ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 -- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 msgr2=0x56541f69be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 -- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 msgr2=0x56541f654ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f654ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 -- 172.21.15.16:0/3400117433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541f43f270 con 0x56541f6579a0 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.564+0000 7f45cd091700 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0522c0 tx=0x7f45bc083310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f05832809f228 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45aeffd700 1 -- 172.21.15.16:0/3400117433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc059070 con 0x56541f6579a0 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45aeffd700 1 -- 172.21.15.16:0/3400117433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc06c1c0 con 0x56541f6579a0 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45aeffd700 1 -- 172.21.15.16:0/3400117433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc05b0f0 con 0x56541f6579a0 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 -- 172.21.15.16:0/3400117433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 msgr2=0x56541f654580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0522c0 tx=0x7f45bc083310 comp rx=0 tx=0).stop 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 -- 172.21.15.16:0/3400117433 shutdown_connections 2023-10-06T11:42:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f654ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/3400117433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 -- 172.21.15.16:0/3400117433 >> 172.21.15.16:0/3400117433 conn(0x56541f656070 msgr2=0x56541f656440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 -- 172.21.15.16:0/3400117433 shutdown_connections 2023-10-06T11:42:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.565+0000 7f45d8deeb80 1 -- 172.21.15.16:0/3400117433 wait complete. 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.566+0000 7f45d8deeb80 1 -- start start 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.566+0000 7f45d8deeb80 1 Processor -- start 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f6ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f654ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f5a5060 con 0x56541f6579a0 2023-10-06T11:42:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f550890 con 0x56541f655bd0 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541f56bea0 con 0x56541f69ba90 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45d8deeb80 10 client.0 _ll_register_callbacks cb 0x56541f441f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37988/0 (socket says 172.21.15.16:37988) 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 -- 172.21.15.16:0/2472302927 learned_addr learned my addr 172.21.15.16:0/2472302927 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd892700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f6ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45ce093700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f654ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 -- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 msgr2=0x56541f654ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f654ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 -- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 msgr2=0x56541f6ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f6ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 -- 172.21.15.16:0/2472302927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c80acf60 con 0x56541f6579a0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45cd091700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45bc052270 tx=0x7f45bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db4379c0e8deb55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.567+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc053fd0 con 0x56541f6579a0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45bc01b300 con 0x56541f6579a0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56541f649690 con 0x56541f6579a0 2023-10-06T11:42:11.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc01b4e0 con 0x56541f6579a0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 10 client.4662 fetch_fsmap learned FSMap version 7 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56541f3e1fe0 con 0x56541f6579a0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56541f649690 con 0x56541f6579a0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f45bc0552b0 con 0x56541f6579a0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 10 client.4662 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 20 client.4662 populate_metadata read hostname 'smithi016' 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f67f7a0 0x56541f696ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 4 client.4662 mds_command: new command op to 4297 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 --> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56541f6265c0 con 0x56541f67f7a0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.568+0000 7f45cd892700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f67f7a0 0x56541f696ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45cd892700 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f67f7a0 0x56541f696ca0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c702037b8e6090d2 server_cookie=80a029e623bf1df7 in_seq=0 out_seq=0 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45a7fff700 10 client.4662 ms_handle_connect on v2:172.21.15.177:6835/1602610299 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45a7fff700 1 -- 172.21.15.16:0/2472302927 <== mds.0 v2:172.21.15.177:6835/1602610299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56541f6265c0 con 0x56541f67f7a0 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45a7fff700 10 client.4662 handle_command_reply: tid=0 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 client.4662 shutdown 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 msgr2=0x56541f654580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45bc052270 tx=0x7f45bc01f000 comp rx=0 tx=0).stop 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 shutdown_connections 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f67f7a0 0x56541f696ca0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f654ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45a7fff700 0 client.4662 ms_handle_reset on v2:172.21.15.177:6835/1602610299 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f655bd0 0x56541f6ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/2472302927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f654580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 >> 172.21.15.16:0/2472302927 conn(0x56541f656070 msgr2=0x56541f651720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 shutdown_connections 2023-10-06T11:42:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.569+0000 7f45d8deeb80 1 -- 172.21.15.16:0/2472302927 wait complete. 2023-10-06T11:42:11.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.570+0000 7f45d8deeb80 20 client.4662 trim_cache size 0 max 16384 2023-10-06T11:42:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.586+0000 7f45d8deeb80 1 Processor -- start 2023-10-06T11:42:11.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.586+0000 7f45d8deeb80 1 -- start start 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f6579a0 0x56541f69de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f427e30 con 0x56541f655bd0 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f6510f0 con 0x56541f6579a0 2023-10-06T11:42:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541f654460 con 0x56541f69ba90 2023-10-06T11:42:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37996/0 (socket says 172.21.15.16:37996) 2023-10-06T11:42:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 -- 172.21.15.16:0/10884620 learned_addr learned my addr 172.21.15.16:0/10884620 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45ce093700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd091700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f6579a0 0x56541f69de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 -- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 msgr2=0x56541f69e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 -- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f6579a0 msgr2=0x56541f69de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f6579a0 0x56541f69de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45cd892700 1 -- 172.21.15.16:0/10884620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56541f573780 con 0x56541f655bd0 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.587+0000 7f45ce093700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45cd892700 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45c0001950 tx=0x7f45c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2af17909fd21a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45af7fe700 1 -- 172.21.15.16:0/10884620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0015e60 con 0x56541f655bd0 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45af7fe700 1 -- 172.21.15.16:0/10884620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0022850 con 0x56541f655bd0 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45af7fe700 1 -- 172.21.15.16:0/10884620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00229e0 con 0x56541f655bd0 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 -- 172.21.15.16:0/10884620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 msgr2=0x56541f6801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45c0001950 tx=0x7f45c0018000 comp rx=0 tx=0).stop 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 -- 172.21.15.16:0/10884620 shutdown_connections 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f655bd0 0x56541f6801c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f69ba90 0x56541f69e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/10884620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f6579a0 0x56541f69de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 -- 172.21.15.16:0/10884620 >> 172.21.15.16:0/10884620 conn(0x56541f656070 msgr2=0x56541f695880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 -- 172.21.15.16:0/10884620 shutdown_connections 2023-10-06T11:42:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.588+0000 7f45d8deeb80 1 -- 172.21.15.16:0/10884620 wait complete. 2023-10-06T11:42:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.589+0000 7f45d8deeb80 1 -- start start 2023-10-06T11:42:11.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.589+0000 7f45d8deeb80 1 Processor -- start 2023-10-06T11:42:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f655bd0 0x56541f651990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 0x56541f6ff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f7005c0 con 0x56541f6579a0 2023-10-06T11:42:11.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56541f67d820 con 0x56541f69ba90 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56541f67da60 con 0x56541f655bd0 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45d8deeb80 10 client.0 _ll_register_callbacks cb 0x56541f441f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38000/0 (socket says 172.21.15.16:38000) 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 -- 172.21.15.16:0/99783228 learned_addr learned my addr 172.21.15.16:0/99783228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd892700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f655bd0 0x56541f651990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45ce093700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 0x56541f6ff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 -- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f655bd0 msgr2=0x56541f651990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f655bd0 0x56541f651990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 -- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 msgr2=0x56541f6ff980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 0x56541f6ff980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 -- 172.21.15.16:0/99783228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c000f040 con 0x56541f6579a0 2023-10-06T11:42:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45ce093700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 0x56541f6ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:11.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45cd091700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45bc055ac0 tx=0x7f45bc06f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b4c201c58a44b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.590+0000 7f45a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.592+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc055900 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.592+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45bc06c1c0 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.592+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f45bc0552b0 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.592+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc05cae0 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.592+0000 7f45d8deeb80 10 client.4668 fetch_fsmap learned FSMap version 7 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56541f55a470 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc06c1c0 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f45bc083270 con 0x56541f6579a0 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 10 client.4668 fetch_fsmap finished waiting for FSMap version 7 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 10 client.4668 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T11:42:11.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 20 client.4668 populate_metadata read hostname 'smithi016' 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f6ffec0 0x56541f67a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 4 client.4668 mds_command: new command op to 4297 tid=0[{"prefix": "status"}] 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 --> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56541f547e00 con 0x56541f6ffec0 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45cd892700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f6ffec0 0x56541f67a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45a7fff700 10 client.4668 ms_handle_connect on v2:172.21.15.177:6835/1602610299 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.593+0000 7f45cd892700 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f6ffec0 0x56541f67a4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bdd54997046b7c2 server_cookie=4b079b0814893bdf in_seq=0 out_seq=0 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45a7fff700 1 -- 172.21.15.16:0/99783228 <== mds.0 v2:172.21.15.177:6835/1602610299 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56541f547e00 con 0x56541f6ffec0 2023-10-06T11:42:11.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45a7fff700 10 client.4668 handle_command_reply: tid=0 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 client.4668 shutdown 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 msgr2=0x56541f6920c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f45bc055ac0 tx=0x7f45bc06f070 comp rx=0 tx=0).stop 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 shutdown_connections 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:6835/1602610299,v1:172.21.15.177:6837/1602610299] conn(0x56541f6ffec0 0x56541f67a4b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56541f69ba90 0x56541f6ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45a7fff700 0 client.4668 ms_handle_reset on v2:172.21.15.177:6835/1602610299 2023-10-06T11:42:11.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x56541f655bd0 0x56541f651990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 --2- 172.21.15.16:0/99783228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56541f6579a0 0x56541f6920c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 >> 172.21.15.16:0/99783228 conn(0x56541f656070 msgr2=0x56541f6548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 shutdown_connections 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 1 -- 172.21.15.16:0/99783228 wait complete. 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.594+0000 7f45d8deeb80 20 client.4668 trim_cache size 0 max 16384 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "b7fe6d1b-a342-44e2-9c53-ba3d26f05e41", 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4297, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 15.073978990000001, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 7, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 21, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T11:42:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 21.033848485 2023-10-06T11:42:11.609 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 msgr2=0x7f45b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 0x7f45b40693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45c00042e0 tx=0x7f45c0001c00 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 msgr2=0x7f45c814a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45bc011960 tx=0x7f45bc008040 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 shutdown_connections 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f45b4066f30 0x7f45b40693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f45c80b0290 0x7f45c80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f45b00051c0 0x7f45c80b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 --2- 172.21.15.16:0/1128757388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45c8149c60 0x7f45c814a0d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 >> 172.21.15.16:0/1128757388 conn(0x7f45c806d520 msgr2=0x7f45c80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 shutdown_connections 2023-10-06T11:42:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.595+0000 7f45cfaf6700 1 -- 172.21.15.16:0/1128757388 wait complete. 2023-10-06T11:42:11.612 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T11:42:11.631 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T11:42:11.631 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T11:42:11.631 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T11:42:11.631 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T11:42:11.631 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T11:42:11.632 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T11:42:11.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:11.632 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T11:42:11.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:11.651 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-10-06T11:42:11.720 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-10-06T11:42:11.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:42:11.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:11.722 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-10-06T11:42:11.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:11.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-10-06T11:42:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2023-10-06T11:42:11.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:42:11.806 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T11:42:11.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:11.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.964+0000 7ff334dc5700 1 Processor -- start 2023-10-06T11:42:11.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.964+0000 7ff334dc5700 1 -- start start 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff334dc5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3301595b0 0x7ff3300cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff334dc5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff3300cf380 0x7ff3300cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff334dc5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300cf780 con 0x7ff330159980 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff334dc5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300cbf20 con 0x7ff3301595b0 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff334dc5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300cc200 con 0x7ff3300cf380 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff32e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3301595b0 0x7ff3300cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff32e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff3301595b0 0x7ff3300cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38008/0 (socket says 172.21.15.16:38008) 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.965+0000 7ff32e59c700 1 -- 172.21.15.16:0/1568360678 learned_addr learned my addr 172.21.15.16:0/1568360678 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32ed9d700 1 --2- 172.21.15.16:0/1568360678 >> v2:172.21.15.177:3300/0 conn(0x7ff3300cf380 0x7ff3300cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32e59c700 1 -- 172.21.15.16:0/1568360678 >> v2:172.21.15.177:3300/0 conn(0x7ff3300cf380 msgr2=0x7ff3300cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32e59c700 1 --2- 172.21.15.16:0/1568360678 >> v2:172.21.15.177:3300/0 conn(0x7ff3300cf380 0x7ff3300cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32e59c700 1 -- 172.21.15.16:0/1568360678 >> v1:172.21.15.177:6789/0 conn(0x7ff330159980 legacy=0x7ff3300cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32e59c700 1 -- 172.21.15.16:0/1568360678 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3300cc4e0 con 0x7ff3301595b0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32e59c700 1 --2- 172.21.15.16:0/1568360678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff3300cd140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3300b3290 tx=0x7ff320009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459717c8eeda0915 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32d59a700 1 -- 172.21.15.16:0/1568360678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000b820 con 0x7ff3301595b0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32d59a700 1 -- 172.21.15.16:0/1568360678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32000b980 con 0x7ff3301595b0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff32d59a700 1 -- 172.21.15.16:0/1568360678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32000bd20 con 0x7ff3301595b0 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff334dc5700 1 -- 172.21.15.16:0/1568360678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 msgr2=0x7ff3300cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.966+0000 7ff334dc5700 1 --2- 172.21.15.16:0/1568360678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff3300cd140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3300b3290 tx=0x7ff320009250 comp rx=0 tx=0).stop 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 -- 172.21.15.16:0/1568360678 shutdown_connections 2023-10-06T11:42:11.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 --2- 172.21.15.16:0/1568360678 >> v2:172.21.15.177:3300/0 conn(0x7ff3300cf380 0x7ff3300cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 --2- 172.21.15.16:0/1568360678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff3300cd140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 -- 172.21.15.16:0/1568360678 >> 172.21.15.16:0/1568360678 conn(0x7ff330099b40 msgr2=0x7ff330087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 -- 172.21.15.16:0/1568360678 shutdown_connections 2023-10-06T11:42:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.967+0000 7ff334dc5700 1 -- 172.21.15.16:0/1568360678 wait complete. 2023-10-06T11:42:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 Processor -- start 2023-10-06T11:42:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 -- start start 2023-10-06T11:42:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3300cf380 0x7ff330151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff330159980 0x7ff3300c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff330157330 con 0x7ff3301595b0 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff330156f00 con 0x7ff330159980 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.968+0000 7ff334dc5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300c8ce0 con 0x7ff3300cf380 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38020/0 (socket says 172.21.15.16:38020) 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 -- 172.21.15.16:0/2472601498 learned_addr learned my addr 172.21.15.16:0/2472601498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:11.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32ed9d700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff330159980 0x7ff3300c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32e59c700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3300cf380 0x7ff330151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 -- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3300cf380 msgr2=0x7ff330151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3300cf380 0x7ff330151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 -- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff330159980 msgr2=0x7ff3300c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff330159980 0x7ff3300c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 -- 172.21.15.16:0/2472601498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3200094f0 con 0x7ff3301595b0 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff32dd9b700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff31800b5e0 tx=0x7ff318009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800f510 con 0x7ff3301595b0 2023-10-06T11:42:11.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800f670 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff330157b70 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800fa30 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.969+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff330157800 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.970+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff330157d00 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.971+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff31801a400 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.971+0000 7ff3267fc700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.971+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff318083fa0 con 0x7ff3301595b0 2023-10-06T11:42:11.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.971+0000 7ff32e59c700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:11.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.972+0000 7ff32e59c700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 0x7ff3100693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3300b3290 tx=0x7ff320016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:11.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:11.975+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff318051130 con 0x7ff3301595b0 2023-10-06T11:42:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.217+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff330152540 con 0x7ff3301595b0 2023-10-06T11:42:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.218+0000 7ff3267fc700 1 -- 172.21.15.16:0/2472601498 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff318054750 con 0x7ff3301595b0 2023-10-06T11:42:12.219 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 msgr2=0x7ff3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 0x7ff3100693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3300b3290 tx=0x7ff320016000 comp rx=0 tx=0).stop 2023-10-06T11:42:12.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 msgr2=0x7ff330169790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff31800b5e0 tx=0x7ff318009120 comp rx=0 tx=0).stop 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 shutdown_connections 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff310066f30 0x7ff3100693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff330159980 0x7ff3300c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3300cf380 0x7ff330151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 --2- 172.21.15.16:0/2472601498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3301595b0 0x7ff330169790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 >> 172.21.15.16:0/2472601498 conn(0x7ff330099b40 msgr2=0x7ff33006f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:12.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 shutdown_connections 2023-10-06T11:42:12.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:12.220+0000 7ff334dc5700 1 -- 172.21.15.16:0/2472601498 wait complete. 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:12.233 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:12.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:12.234 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:42:12.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.234 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:12.249 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-10-06T11:42:12.250 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42462sec preferred_lft 42462sec 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:12.251 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-10-06T11:42:12.251 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:42:12.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:42:12.252 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:12.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.406 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:12.451 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-10-06T11:42:12.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.453 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-10-06T11:42:12.487 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-10-06T11:42:12.488 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:12.488 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T11:42:12.488 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T11:42:12.488 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:12.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.489 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:12.489 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:12.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:42:12.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:42:12.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:42:12.490 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.739 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-10-06T11:42:12.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.756 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-10-06T11:42:12.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.810 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:12.810 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:12.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:42:12.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T11:42:12.810 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:12.900 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T11:42:12.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T11:42:12.901 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:13.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:13.096 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:13.096 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:13.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-10-06T11:42:13.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:42:13.096 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:13.189 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T11:42:13.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:13.190 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:13.204 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:42:13.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:13.205 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:13.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:13.325 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:13.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:13.394 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:13.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:13.395 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:13.450 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:42:13.450 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T11:42:13.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:13.517 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:13.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:13.582 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:13.583 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:13.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:13.584 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:13.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-10-06T11:42:13.584+0000 7f9212ce2540 -1 init, newargv = 0x5599f8a2ca90 newargc=15 2023-10-06T11:42:13.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[99320]: starting ceph client 2023-10-06T11:42:13.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[99320]: starting fuse 2023-10-06T11:42:13.600 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-10-06T11:42:13.600 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T11:42:13.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:13.600 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-10-06T11:42:13.601 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-10-06T11:42:13.602 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-10-06T11:42:13.723 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.99320.asok 2023-10-06T11:42:13.905 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99320.asok 2023-10-06T11:42:13.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:13.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99320.asok status 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "99320", 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:14.061 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4680, 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4680 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:14.062 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2632851964 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2632851964 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4680 192.168.0.1:0/2632851964", 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/2632851964", 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 7, 2023-10-06T11:42:14.063 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 21, 2023-10-06T11:42:14.064 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:14.064 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-10-06T11:42:14.064 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-10-06T11:42:14.064 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:14.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.189+0000 7ffb31274700 1 Processor -- start 2023-10-06T11:42:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.189+0000 7ffb31274700 1 -- start start 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.189+0000 7ffb31274700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffb2c1595b0 0x7ffb2c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb31274700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c0cf780 con 0x7ffb2c0cf380 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb31274700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c0cbf20 con 0x7ffb2c159980 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb31274700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c0cc200 con 0x7ffb2c1595b0 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb2c0cf380 0x7ffb2c0c5610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35938/0 (socket says 172.21.15.16:35938) 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2b7fe700 1 -- 172.21.15.16:0/3046694461 learned_addr learned my addr 172.21.15.16:0/3046694461 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2affd700 1 --2- 172.21.15.16:0/3046694461 >> v2:172.21.15.177:3300/0 conn(0x7ffb2c1595b0 0x7ffb2c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb29ffb700 1 -- 172.21.15.16:0/3046694461 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056803866 0 0) 0x7ffb2c0cf780 con 0x7ffb2c0cf380 2023-10-06T11:42:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb29ffb700 1 -- 172.21.15.16:0/3046694461 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb14003570 con 0x7ffb2c0cf380 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2affd700 1 -- 172.21.15.16:0/3046694461 >> v1:172.21.15.177:6789/0 conn(0x7ffb2c159980 legacy=0x7ffb2c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2affd700 1 -- 172.21.15.16:0/3046694461 >> v1:172.21.15.16:6789/0 conn(0x7ffb2c0cf380 legacy=0x7ffb2c0c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2affd700 1 -- 172.21.15.16:0/3046694461 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0cc4e0 con 0x7ffb2c1595b0 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.190+0000 7ffb2affd700 1 --2- 172.21.15.16:0/3046694461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0cd140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0098e0 tx=0x7ffb1c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=feee4af5a14164be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb29ffb700 1 -- 172.21.15.16:0/3046694461 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00b640 con 0x7ffb2c1595b0 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb29ffb700 1 -- 172.21.15.16:0/3046694461 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c00b7a0 con 0x7ffb2c1595b0 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb29ffb700 1 -- 172.21.15.16:0/3046694461 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00bb40 con 0x7ffb2c1595b0 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 -- 172.21.15.16:0/3046694461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c1595b0 msgr2=0x7ffb2c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 --2- 172.21.15.16:0/3046694461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0cd140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0098e0 tx=0x7ffb1c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 -- 172.21.15.16:0/3046694461 shutdown_connections 2023-10-06T11:42:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 --2- 172.21.15.16:0/3046694461 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0cd140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 -- 172.21.15.16:0/3046694461 >> 172.21.15.16:0/3046694461 conn(0x7ffb2c099b40 msgr2=0x7ffb2c087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 -- 172.21.15.16:0/3046694461 shutdown_connections 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.191+0000 7ffb31274700 1 -- 172.21.15.16:0/3046694461 wait complete. 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.192+0000 7ffb31274700 1 Processor -- start 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.192+0000 7ffb31274700 1 -- start start 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c0cf380 0x7ffb2c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c159980 0x7ffb2c0ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c157960 con 0x7ffb2c1595b0 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c156d50 con 0x7ffb2c159980 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb31274700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c157030 con 0x7ffb2c0cf380 2023-10-06T11:42:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb2a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb2a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38032/0 (socket says 172.21.15.16:38032) 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb2a7fc700 1 -- 172.21.15.16:0/3220470133 learned_addr learned my addr 172.21.15.16:0/3220470133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb2b7fe700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c159980 0x7ffb2c0ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.193+0000 7ffb2affd700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c0cf380 0x7ffb2c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 -- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c0cf380 msgr2=0x7ffb2c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c0cf380 0x7ffb2c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 -- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c159980 msgr2=0x7ffb2c0ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c159980 0x7ffb2c0ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 -- 172.21.15.16:0/3220470133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c009420 con 0x7ffb2c1595b0 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb2a7fc700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c4e0 tx=0x7ffb2000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011590 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20011710 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c157310 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb20011ad0 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.194+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c1532b0 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.195+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c154650 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.195+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb2001b3b0 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.196+0000 7ffb12ffd700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 0x7ffb180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.196+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffb20083a20 con 0x7ffb2c1595b0 2023-10-06T11:42:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.196+0000 7ffb2affd700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 0x7ffb180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.196+0000 7ffb2affd700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 0x7ffb180693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0097b0 tx=0x7ffb1c017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.200+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb20050b30 con 0x7ffb2c1595b0 2023-10-06T11:42:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.440+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb2c04c380 con 0x7ffb2c1595b0 2023-10-06T11:42:14.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.441+0000 7ffb12ffd700 1 -- 172.21.15.16:0/3220470133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb20017070 con 0x7ffb2c1595b0 2023-10-06T11:42:14.442 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 msgr2=0x7ffb180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 0x7ffb180693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0097b0 tx=0x7ffb1c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 msgr2=0x7ffb2c0ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000c4e0 tx=0x7ffb2000c7f0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 shutdown_connections 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ffb18066f30 0x7ffb180693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffb2c159980 0x7ffb2c0ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffb2c0cf380 0x7ffb2c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 --2- 172.21.15.16:0/3220470133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb2c1595b0 0x7ffb2c0ca170 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 >> 172.21.15.16:0/3220470133 conn(0x7ffb2c099b40 msgr2=0x7ffb2c06f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 shutdown_connections 2023-10-06T11:42:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:14.443+0000 7ffb31274700 1 -- 172.21.15.16:0/3220470133 wait complete. 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:14.456 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:14.457 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:14.457 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:42:14.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.457 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-10-06T11:42:14.472 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:14.472 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:14.472 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:14.473 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42459sec preferred_lft 42459sec 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:dd:72:ae:41:b0 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:14.474 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::582b:4fff:fe01:ca4e/64 scope link 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:dd:72:ae:41:b0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::24dd:72ff:feae:41b0/64 scope link tentative 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.475 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:14.536 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-10-06T11:42:14.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.538 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-10-06T11:42:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-10-06T11:42:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T11:42:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T11:42:14.595 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:14.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:42:14.596 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:14.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.697 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-10-06T11:42:14.712 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:42:14.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.712 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-10-06T11:42:14.766 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T11:42:14.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.767 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:14.767 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:14.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:42:14.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T11:42:14.767 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:14.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fa:3b:2e:34:eb:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:42:14.894 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:42:14.895 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.895 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f83b:2eff:fe34:ebaf/64 scope link tentative 2023-10-06T11:42:14.895 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:14.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fa:3b:2e:34:eb:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:14.987 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f83b:2eff:fe34:ebaf/64 scope link tentative 2023-10-06T11:42:14.988 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:15.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T11:42:15.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:15.006 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T11:42:15.007 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:15.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:15.221 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:15.222 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:15.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2023-10-06T11:42:15.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:42:15.222 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:15.315 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T11:42:15.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:15.316 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:15.331 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T11:42:15.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:15.331 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:15.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:15.451 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:15.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:15.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:15.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:15.518 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-10-06T11:42:15.573 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-10-06T11:42:15.573 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T11:42:15.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:15.640 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:15.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:15.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2023-10-06T11:42:15.698+0000 7f9d8f289540 -1 init, newargv = 0x564df4d93980 newargc=15 2023-10-06T11:42:15.700 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[99853]: starting ceph client 2023-10-06T11:42:15.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[99853]: starting fuse 2023-10-06T11:42:15.706 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:15.707 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:15.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:15.708 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:15.763 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-10-06T11:42:15.763 INFO:teuthology.orchestra.run.smithi016.stdout:48 2023-10-06T11:42:15.763 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-10-06T11:42:15.764 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-10-06T11:42:15.765 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-10-06T11:42:15.766 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:15.766 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:15.766 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2023-10-06T11:42:15.766 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-10-06T11:42:15.886 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.99853.asok 2023-10-06T11:42:16.068 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.99853.asok 2023-10-06T11:42:16.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:16.068 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.99853.asok status 2023-10-06T11:42:16.222 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "99853", 2023-10-06T11:42:16.223 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4692, 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4692 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:16.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 372170800 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 372170800 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4692 172.21.15.16:0/372170800", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "172.21.15.16:0/372170800", 2023-10-06T11:42:16.225 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 7, 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 21, 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-10-06T11:42:16.226 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:16.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.351+0000 7f35250f7700 1 Processor -- start 2023-10-06T11:42:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.351+0000 7f35250f7700 1 -- start start 2023-10-06T11:42:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f35250f7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35201595b0 0x7f35200cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f35250f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 0x7f35200cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f35250f7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35200cbf20 con 0x7f35200cec70 2023-10-06T11:42:16.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f35250f7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200cc200 con 0x7f35201595b0 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f35250f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35200cc4e0 con 0x7f3520159980 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35201595b0 0x7f35200cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35201595b0 0x7f35200cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38038/0 (socket says 172.21.15.16:38038) 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 -- 172.21.15.16:0/3017413536 learned_addr learned my addr 172.21.15.16:0/3017413536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351e59c700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 0x7f35200cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 -- 172.21.15.16:0/3017413536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 msgr2=0x7f35200cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 0x7f35200cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 -- 172.21.15.16:0/3017413536 >> v1:172.21.15.177:6789/0 conn(0x7f35200cec70 legacy=0x7f35200cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 -- 172.21.15.16:0/3017413536 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200cc7c0 con 0x7f35201595b0 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351e59c700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 0x7f35200cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351ed9d700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f35200cd140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35100099b0 tx=0x7f3510009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de6f335f338a896 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351dd9b700 1 -- 172.21.15.16:0/3017413536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000b860 con 0x7f35201595b0 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351dd9b700 1 -- 172.21.15.16:0/3017413536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000b9c0 con 0x7f35201595b0 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.352+0000 7f351dd9b700 1 -- 172.21.15.16:0/3017413536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000bd60 con 0x7f35201595b0 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 -- 172.21.15.16:0/3017413536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 msgr2=0x7f35200cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f35200cd140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35100099b0 tx=0x7f3510009250 comp rx=0 tx=0).stop 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 -- 172.21.15.16:0/3017413536 shutdown_connections 2023-10-06T11:42:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f35200cd140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 --2- 172.21.15.16:0/3017413536 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3520159980 0x7f35200cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 -- 172.21.15.16:0/3017413536 >> 172.21.15.16:0/3017413536 conn(0x7f3520099b40 msgr2=0x7f3520087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 -- 172.21.15.16:0/3017413536 shutdown_connections 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.353+0000 7f35250f7700 1 -- 172.21.15.16:0/3017413536 wait complete. 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.354+0000 7f35250f7700 1 Processor -- start 2023-10-06T11:42:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.354+0000 7f35250f7700 1 -- start start 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35200cec70 0x7f352014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3520159980 0x7f3520143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520155d60 con 0x7f35201595b0 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352016cfc0 con 0x7f3520159980 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f35250f7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3520155a10 con 0x7f35200cec70 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f351e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f351e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38050/0 (socket says 172.21.15.16:38050) 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f351e59c700 1 -- 172.21.15.16:0/1713946488 learned_addr learned my addr 172.21.15.16:0/1713946488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f351ed9d700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35200cec70 0x7f352014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.355+0000 7f351f59e700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3520159980 0x7f3520143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 -- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35200cec70 msgr2=0x7f352014f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35200cec70 0x7f352014f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 -- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3520159980 msgr2=0x7f3520143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3520159980 0x7f3520143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 -- 172.21.15.16:0/1713946488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35100094f0 con 0x7f35201595b0 2023-10-06T11:42:16.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f351e59c700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f351400d7e0 tx=0x7f3514008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35140125f0 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3514012750 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514012b10 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352016d260 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.356+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352016d820 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.357+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352004c380 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.357+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3514013480 con 0x7f35201595b0 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.358+0000 7f3506ffd700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 0x7f350c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:16.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.358+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f35140849f0 con 0x7f35201595b0 2023-10-06T11:42:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.358+0000 7f351ed9d700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 0x7f350c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.358+0000 7f351ed9d700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 0x7f350c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35100099b0 tx=0x7f3510017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:16.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.362+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3514051b00 con 0x7f35201595b0 2023-10-06T11:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.601+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3520155ea0 con 0x7f35201595b0 2023-10-06T11:42:16.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.601+0000 7f3506ffd700 1 -- 172.21.15.16:0/1713946488 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3514055120 con 0x7f35201595b0 2023-10-06T11:42:16.602 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 msgr2=0x7f350c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 0x7f350c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f35100099b0 tx=0x7f3510017000 comp rx=0 tx=0).stop 2023-10-06T11:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 msgr2=0x7f352016ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:16.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f351400d7e0 tx=0x7f3514008040 comp rx=0 tx=0).stop 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 shutdown_connections 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f350c066f30 0x7f350c0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3520159980 0x7f3520143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35200cec70 0x7f352014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 --2- 172.21.15.16:0/1713946488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35201595b0 0x7f352016ca80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 >> 172.21.15.16:0/1713946488 conn(0x7f3520099b40 msgr2=0x7f35201516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 shutdown_connections 2023-10-06T11:42:16.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:16.603+0000 7f35250f7700 1 -- 172.21.15.16:0/1713946488 wait complete. 2023-10-06T11:42:16.616 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:16.617 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:42:16.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:16.618 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-10-06T11:42:16.636 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:16.636 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:16.636 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:16.636 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:16.637 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42462sec preferred_lft 42462sec 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:16.638 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T11:42:16.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T11:42:16.639 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:16.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:16.807 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:16.850 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T11:42:16.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:16.851 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:16.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T11:42:16.929 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:17.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:17.160 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T11:42:17.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:17.175 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-10-06T11:42:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:17.228 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:17.228 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:17.228 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:42:17.229 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T11:42:17.229 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:17.316 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T11:42:17.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T11:42:17.317 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:17.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:17.526 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:17.526 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:17.526 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2023-10-06T11:42:17.526 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T11:42:17.526 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:17.618 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T11:42:17.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:17.618 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:17.633 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:42:17.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:17.633 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:17.687 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:17.751 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:17.751 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:17.816 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:17.817 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:17.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:17.817 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:17.872 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:42:17.872 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T11:42:17.914 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:17.938 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:17.938 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:17.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:2023-10-06T11:42:17.997+0000 7f17a33df540 -1 init, newargv = 0x562e48ba0b70 newargc=15 2023-10-06T11:42:17.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[97101]: starting ceph client 2023-10-06T11:42:18.003 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:18.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[97101]: starting fuse 2023-10-06T11:42:18.004 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:18.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:18.005 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:18.059 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T11:42:18.059 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> import glob 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> import re 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> import os 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> import subprocess 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> def _find_admin_socket(client_name): 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> files = glob.glob(asok_path) 2023-10-06T11:42:18.060 DEBUG:teuthology.orchestra.run.smithi177:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> # Given a non-glob path, it better be there 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> if "*" not in asok_path: 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> assert(len(files) == 1) 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> return files[0] 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> for f in files: 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:18.061 DEBUG:teuthology.orchestra.run.smithi177:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> contents = proc_f.read() 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> if mountpoint in contents: 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> return f 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> print(_find_admin_socket("client.1")) 2023-10-06T11:42:18.062 DEBUG:teuthology.orchestra.run.smithi177:> ' 2023-10-06T11:42:18.181 INFO:teuthology.orchestra.run.smithi177.stdout:/var/run/ceph/ceph-client.1.97101.asok 2023-10-06T11:42:18.363 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.97101.asok 2023-10-06T11:42:18.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:18.364 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.97101.asok status 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "metadata": { 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "entity_id": "1", 2023-10-06T11:42:18.515 INFO:teuthology.orchestra.run.smithi177.stdout: "hostname": "smithi177", 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "pid": "97101", 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "root": "/" 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_count": 0, 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "id": 4669, 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "inst": { 2023-10-06T11:42:18.516 INFO:teuthology.orchestra.run.smithi177.stdout: "name": { 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "client", 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "num": 4669 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2772694079 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:18.517 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2772694079 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "inst_str": "client.4669 192.168.0.1:0/2772694079", 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "addr_str": "192.168.0.1:0/2772694079", 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "inode_count": 1, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_epoch": 7, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch": 21, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklisted": false, 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout: "fs_name": "cephfs" 2023-10-06T11:42:18.518 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T11:42:18.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.657+0000 7f941797f700 1 Processor -- start 2023-10-06T11:42:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.657+0000 7f941797f700 1 -- start start 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941797f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94100ca0c0 0x7f94100a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941797f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941006b430 con 0x7f9410159a10 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941797f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941006b6e0 con 0x7f94100c7370 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941797f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100abc10 con 0x7f94100ca0c0 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f9414f1a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94100ca0c0 0x7f94100a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941571b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9410159a10 0x7f941006ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48452/0 (socket says 172.21.15.16:48452) 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f941571b700 1 -- 172.21.15.16:0/2666952294 learned_addr learned my addr 172.21.15.16:0/2666952294 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f9414f1a700 1 -- 172.21.15.16:0/2666952294 >> v1:172.21.15.177:6789/0 conn(0x7f94100c7370 legacy=0x7f94100c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f9414f1a700 1 -- 172.21.15.16:0/2666952294 >> v1:172.21.15.16:6789/0 conn(0x7f9410159a10 legacy=0x7f941006ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f9414f1a700 1 -- 172.21.15.16:0/2666952294 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941006c5b0 con 0x7f94100ca0c0 2023-10-06T11:42:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.658+0000 7f9407fff700 1 -- 172.21.15.16:0/2666952294 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581145418 0 0) 0x7f941006b430 con 0x7f9410159a10 2023-10-06T11:42:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f9414f1a700 1 --2- 172.21.15.16:0/2666952294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f94100a98b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f940c0098e0 tx=0x7f940c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f916e9c3ba42b04c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f9407fff700 1 -- 172.21.15.16:0/2666952294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c009e80 con 0x7f94100ca0c0 2023-10-06T11:42:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f9407fff700 1 -- 172.21.15.16:0/2666952294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c00b860 con 0x7f94100ca0c0 2023-10-06T11:42:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f9407fff700 1 -- 172.21.15.16:0/2666952294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00bbe0 con 0x7f94100ca0c0 2023-10-06T11:42:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f941797f700 1 -- 172.21.15.16:0/2666952294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 msgr2=0x7f94100a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f941797f700 1 --2- 172.21.15.16:0/2666952294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f94100a98b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f940c0098e0 tx=0x7f940c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f941797f700 1 -- 172.21.15.16:0/2666952294 shutdown_connections 2023-10-06T11:42:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f941797f700 1 --2- 172.21.15.16:0/2666952294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f94100a98b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.659+0000 7f941797f700 1 -- 172.21.15.16:0/2666952294 >> 172.21.15.16:0/2666952294 conn(0x7f94100cd690 msgr2=0x7f94100c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.660+0000 7f941797f700 1 -- 172.21.15.16:0/2666952294 shutdown_connections 2023-10-06T11:42:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.660+0000 7f941797f700 1 -- 172.21.15.16:0/2666952294 wait complete. 2023-10-06T11:42:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.661+0000 7f941797f700 1 Processor -- start 2023-10-06T11:42:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.661+0000 7f941797f700 1 -- start start 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94100c7370 0x7f941014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 0x7f941016cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941015cf40 con 0x7f94100ca0c0 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100abd50 con 0x7f9410159a10 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941015a790 con 0x7f94100c7370 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57818/0 (socket says 172.21.15.16:57818) 2023-10-06T11:42:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 -- 172.21.15.16:0/2356682330 learned_addr learned my addr 172.21.15.16:0/2356682330 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941571b700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94100c7370 0x7f941014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9415f1c700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 0x7f941016cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 -- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94100c7370 msgr2=0x7f941014e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94100c7370 0x7f941014e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 -- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 msgr2=0x7f941016cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 0x7f941016cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 -- 172.21.15.16:0/2356682330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c009420 con 0x7f94100ca0c0 2023-10-06T11:42:18.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9415f1c700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 0x7f941016cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f9414f1a700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b3c0 tx=0x7f940c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c01f070 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c011420 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941015ac20 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c00bbe0 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.662+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100abfa0 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.663+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941004c380 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.664+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f940c011a50 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.664+0000 7f94057fa700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 0x7f93f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.664+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f940c085cc0 con 0x7f94100ca0c0 2023-10-06T11:42:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.664+0000 7f941571b700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 0x7f93f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.665+0000 7f941571b700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 0x7f93f40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94100cc5f0 tx=0x7f9400009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.669+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f940c08c080 con 0x7f94100ca0c0 2023-10-06T11:42:18.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.911+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9410070700 con 0x7f94100ca0c0 2023-10-06T11:42:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.911+0000 7f94057fa700 1 -- 172.21.15.16:0/2356682330 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f940c015060 con 0x7f94100ca0c0 2023-10-06T11:42:18.912 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 msgr2=0x7f93f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 0x7f93f40693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94100cc5f0 tx=0x7f9400009040 comp rx=0 tx=0).stop 2023-10-06T11:42:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 msgr2=0x7f9410157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f940c00b3c0 tx=0x7f940c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 shutdown_connections 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f93f4066f30 0x7f93f40693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9410159a10 0x7f941016cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94100c7370 0x7f941014e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 --2- 172.21.15.16:0/2356682330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94100ca0c0 0x7f9410157f20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 >> 172.21.15.16:0/2356682330 conn(0x7f94100cd690 msgr2=0x7f94100c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 shutdown_connections 2023-10-06T11:42:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:18.913+0000 7f941797f700 1 -- 172.21.15.16:0/2356682330 wait complete. 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:18.926 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:18.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:18.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:18.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:42:18.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:18.928 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-10-06T11:42:18.943 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:18.943 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:18.943 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:18.943 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:18.944 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42460sec preferred_lft 42460sec 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2023-10-06T11:42:18.945 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.946 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:18.946 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether de:02:13:dd:df:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:18.946 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:42:18.946 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.946 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::3805:bdff:fe6b:f2aa/64 scope link 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether de:02:13:dd:df:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::dc02:13ff:fedd:df15/64 scope link 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:18.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:18.947 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:19.007 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T11:42:19.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.009 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-10-06T11:42:19.065 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-10-06T11:42:19.065 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:19.065 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T11:42:19.066 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T11:42:19.066 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:19.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.066 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:19.066 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:19.066 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T11:42:19.066 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T11:42:19.067 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T11:42:19.067 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:19.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.165 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T11:42:19.180 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:42:19.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.180 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-10-06T11:42:19.236 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T11:42:19.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.237 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:19.237 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:19.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:42:19.237 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T11:42:19.238 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:19.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.324 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 3e:ea:89:9e:7a:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:42:19.358 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:42:19.359 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.359 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::3cea:89ff:fe9e:7ad1/64 scope link 2023-10-06T11:42:19.359 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.367 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.441 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:19.442 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 3e:ea:89:9e:7a:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T11:42:19.442 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T11:42:19.442 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.442 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::3cea:89ff:fe9e:7ad1/64 scope link 2023-10-06T11:42:19.442 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:19.453 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T11:42:19.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T11:42:19.454 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:19.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:19.684 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:19.684 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:19.684 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2023-10-06T11:42:19.684 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T11:42:19.684 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:19.775 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T11:42:19.775 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:19.776 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:19.790 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T11:42:19.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:19.790 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:19.844 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:19.908 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:19.908 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:19.972 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:19.972 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:19.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:19.973 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:20.027 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T11:42:20.027 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T11:42:20.028 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T11:42:20.070 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:20.094 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:20.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:20.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:2023-10-06T11:42:20.148+0000 7fd024f46540 -1 init, newargv = 0x55c32fb24a70 newargc=15 2023-10-06T11:42:20.153 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:ceph-fuse[97604]: starting ceph client 2023-10-06T11:42:20.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:ceph-fuse[97604]: starting fuse 2023-10-06T11:42:20.159 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:20.160 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:20.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.160 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:20.216 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T11:42:20.216 INFO:teuthology.orchestra.run.smithi177.stdout:48 2023-10-06T11:42:20.216 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T11:42:20.216 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:20.216 DEBUG:teuthology.orchestra.run.smithi177:> import glob 2023-10-06T11:42:20.216 DEBUG:teuthology.orchestra.run.smithi177:> import re 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> import os 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> import subprocess 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> def _find_admin_socket(client_name): 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> files = glob.glob(asok_path) 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> # Given a non-glob path, it better be there 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> if "*" not in asok_path: 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> assert(len(files) == 1) 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> return files[0] 2023-10-06T11:42:20.217 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> for f in files: 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> contents = proc_f.read() 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> if mountpoint in contents: 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> return f 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> print(_find_admin_socket("client.3")) 2023-10-06T11:42:20.218 DEBUG:teuthology.orchestra.run.smithi177:> ' 2023-10-06T11:42:20.336 INFO:teuthology.orchestra.run.smithi177.stdout:/var/run/ceph/ceph-client.3.97604.asok 2023-10-06T11:42:20.519 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.97604.asok 2023-10-06T11:42:20.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.519 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.97604.asok status 2023-10-06T11:42:20.670 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "metadata": { 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "entity_id": "3", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "hostname": "smithi177", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "pid": "97604", 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "root": "/" 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_count": 0, 2023-10-06T11:42:20.671 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "id": 4678, 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "inst": { 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "name": { 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "client", 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "num": 4678 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3983328510 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:20.672 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.2:0", 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3983328510 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "inst_str": "client.4678 192.168.0.2:0/3983328510", 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "addr_str": "192.168.0.2:0/3983328510", 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "inode_count": 1, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_epoch": 7, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch": 21, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklisted": false, 2023-10-06T11:42:20.673 INFO:teuthology.orchestra.run.smithi177.stdout: "fs_name": "cephfs" 2023-10-06T11:42:20.674 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T11:42:20.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.680 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-10-06T11:42:20.680 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:20.699 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-10-06T11:42:20.699 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:20.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:20.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.772 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-10-06T11:42:20.772 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:20.829 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-10-06T11:42:20.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:20.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:20.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.900 DEBUG:teuthology.orchestra.run.smithi177:> stat --file-system '--printf=%T 2023-10-06T11:42:20.900 DEBUG:teuthology.orchestra.run.smithi177:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:20.917 INFO:teuthology.orchestra.run.smithi177.stdout:fuseblk 2023-10-06T11:42:20.918 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:20.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.918 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:20.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:20.982 DEBUG:teuthology.orchestra.run.smithi177:> stat --file-system '--printf=%T 2023-10-06T11:42:20.983 DEBUG:teuthology.orchestra.run.smithi177:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:21.038 INFO:teuthology.orchestra.run.smithi177.stdout:fuseblk 2023-10-06T11:42:21.038 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:21.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:21.038 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:21.102 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-06T11:42:21.113 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T11:42:21.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.255+0000 7f9d559ed700 1 Processor -- start 2023-10-06T11:42:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.255+0000 7f9d559ed700 1 -- start start 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d559ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d559ed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d500ce880 con 0x7f9d500cccf0 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d559ed700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d500ad580 con 0x7f9d5006cbf0 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d559ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5006cfc0 con 0x7f9d501592c0 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d4f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9d500cccf0 0x7f9d500cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48454/0 (socket says 172.21.15.16:48454) 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d4f7fe700 1 -- 172.21.15.16:0/3998543065 learned_addr learned my addr 172.21.15.16:0/3998543065 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.256+0000 7f9d4effd700 1 --2- 172.21.15.16:0/3998543065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570968590 0 0) 0x7f9d500ce880 con 0x7f9d500cccf0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d3c003570 con 0x7f9d500cccf0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499302948 0 0) 0x7f9d500ad580 con 0x7f9d5006cbf0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d500ce880 con 0x7f9d5006cbf0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4effd700 1 -- 172.21.15.16:0/3998543065 >> v1:172.21.15.177:6789/0 conn(0x7f9d5006cbf0 legacy=0x7f9d500cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4effd700 1 -- 172.21.15.16:0/3998543065 >> v1:172.21.15.16:6789/0 conn(0x7f9d500cccf0 legacy=0x7f9d500cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4effd700 1 -- 172.21.15.16:0/3998543065 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d500cbd00 con 0x7f9d501592c0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4effd700 1 --2- 172.21.15.16:0/3998543065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50159690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009f10 tx=0x7f9d40009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac36c6cfcfcf1dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000e070 con 0x7f9d501592c0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000bb40 con 0x7f9d501592c0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.257+0000 7f9d4dffb700 1 -- 172.21.15.16:0/3998543065 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40012430 con 0x7f9d501592c0 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 -- 172.21.15.16:0/3998543065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 msgr2=0x7f9d50159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 --2- 172.21.15.16:0/3998543065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50159690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009f10 tx=0x7f9d40009780 comp rx=0 tx=0).stop 2023-10-06T11:42:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 -- 172.21.15.16:0/3998543065 shutdown_connections 2023-10-06T11:42:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 --2- 172.21.15.16:0/3998543065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50159690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 -- 172.21.15.16:0/3998543065 >> 172.21.15.16:0/3998543065 conn(0x7f9d5008a020 msgr2=0x7f9d50099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 -- 172.21.15.16:0/3998543065 shutdown_connections 2023-10-06T11:42:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.258+0000 7f9d559ed700 1 -- 172.21.15.16:0/3998543065 wait complete. 2023-10-06T11:42:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.259+0000 7f9d559ed700 1 Processor -- start 2023-10-06T11:42:21.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.259+0000 7f9d559ed700 1 -- start start 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d500cccf0 0x7f9d501518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500c75e0 con 0x7f9d5006cbf0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500c78c0 con 0x7f9d500cccf0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500c7d50 con 0x7f9d501592c0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57826/0 (socket says 172.21.15.16:57826) 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 -- 172.21.15.16:0/2895029826 learned_addr learned my addr 172.21.15.16:0/2895029826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4f7fe700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4e7fc700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d500cccf0 0x7f9d501518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 -- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 msgr2=0x7f9d50151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 -- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d500cccf0 msgr2=0x7f9d501518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d500cccf0 0x7f9d501518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 -- 172.21.15.16:0/2895029826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d40009a20 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d4effd700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d500884c0 tx=0x7f9d40019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40012c00 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d40012d80 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d400266d0 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.260+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d500c8060 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.261+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d500c85e0 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.261+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5004c380 con 0x7f9d5006cbf0 2023-10-06T11:42:21.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.262+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d4001c400 con 0x7f9d5006cbf0 2023-10-06T11:42:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.262+0000 7f9d46ffd700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.262+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9d40086440 con 0x7f9d5006cbf0 2023-10-06T11:42:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.263+0000 7f9d4e7fc700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.263+0000 7f9d4e7fc700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 0x7f9d300693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d500cff00 tx=0x7f9d38007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.267+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d40053550 con 0x7f9d5006cbf0 2023-10-06T11:42:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.459+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d500cf970 con 0x7f9d5006cbf0 2023-10-06T11:42:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.460+0000 7f9d46ffd700 1 -- 172.21.15.16:0/2895029826 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f9d40056b70 con 0x7f9d5006cbf0 2023-10-06T11:42:21.461 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:21.462 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 msgr2=0x7f9d300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 0x7f9d300693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9d500cff00 tx=0x7f9d38007d50 comp rx=0 tx=0).stop 2023-10-06T11:42:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 msgr2=0x7f9d500c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d500884c0 tx=0x7f9d40019000 comp rx=0 tx=0).stop 2023-10-06T11:42:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 shutdown_connections 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d5006cbf0 0x7f9d500c9c60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9d30066f30 0x7f9d300693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9d501592c0 0x7f9d50151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 --2- 172.21.15.16:0/2895029826 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d500cccf0 0x7f9d501518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 >> 172.21.15.16:0/2895029826 conn(0x7f9d5008a020 msgr2=0x7f9d50154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 shutdown_connections 2023-10-06T11:42:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.462+0000 7f9d559ed700 1 -- 172.21.15.16:0/2895029826 wait complete. 2023-10-06T11:42:21.476 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:21.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.598+0000 7fb2acfef700 1 Processor -- start 2023-10-06T11:42:21.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.598+0000 7fb2acfef700 1 -- start start 2023-10-06T11:42:21.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2acfef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2a80cef40 0x7fb2a80ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2acfef700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 0x7fb2a80cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2acfef700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a80cffc0 con 0x7fb2a81594a0 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2acfef700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cf3b0 con 0x7fb2a80cef40 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2acfef700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cf690 con 0x7fb2a80c64c0 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2a80cef40 0x7fb2a80ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2a80cef40 0x7fb2a80ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57828/0 (socket says 172.21.15.16:57828) 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/3763880638 learned_addr learned my addr 172.21.15.16:0/3763880638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a7fff700 1 --2- 172.21.15.16:0/3763880638 >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 0x7fb2a80cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/3763880638 >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 msgr2=0x7fb2a80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 --2- 172.21.15.16:0/3763880638 >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 0x7fb2a80cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/3763880638 >> v1:172.21.15.177:6789/0 conn(0x7fb2a81594a0 legacy=0x7fb2a80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/3763880638 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80cf970 con 0x7fb2a80cef40 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.599+0000 7fb2a7fff700 1 --2- 172.21.15.16:0/3763880638 >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 0x7fb2a80cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2a6ffd700 1 --2- 172.21.15.16:0/3763880638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80ccf60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80cde30 tx=0x7fb29c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2afa027a5bbda1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2a67fc700 1 -- 172.21.15.16:0/3763880638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00b860 con 0x7fb2a80cef40 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2a67fc700 1 -- 172.21.15.16:0/3763880638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c00b9c0 con 0x7fb2a80cef40 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2a67fc700 1 -- 172.21.15.16:0/3763880638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00bd60 con 0x7fb2a80cef40 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 -- 172.21.15.16:0/3763880638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 msgr2=0x7fb2a80ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 --2- 172.21.15.16:0/3763880638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80ccf60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80cde30 tx=0x7fb29c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 -- 172.21.15.16:0/3763880638 shutdown_connections 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 --2- 172.21.15.16:0/3763880638 >> v2:172.21.15.177:3300/0 conn(0x7fb2a80c64c0 0x7fb2a80cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 --2- 172.21.15.16:0/3763880638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80ccf60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 -- 172.21.15.16:0/3763880638 >> 172.21.15.16:0/3763880638 conn(0x7fb2a8089f90 msgr2=0x7fb2a80998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:21.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 -- 172.21.15.16:0/3763880638 shutdown_connections 2023-10-06T11:42:21.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.600+0000 7fb2acfef700 1 -- 172.21.15.16:0/3763880638 wait complete. 2023-10-06T11:42:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 Processor -- start 2023-10-06T11:42:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 -- start start 2023-10-06T11:42:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2a80c64c0 0x7fb2a8152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2a81594a0 0x7fb2a80b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8153030 con 0x7fb2a80cef40 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a815baf0 con 0x7fb2a80c64c0 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2acfef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a81538b0 con 0x7fb2a81594a0 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.602+0000 7fb2a6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57840/0 (socket says 172.21.15.16:57840) 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/1815820676 learned_addr learned my addr 172.21.15.16:0/1815820676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a77fe700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2a80c64c0 0x7fb2a8152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a7fff700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2a81594a0 0x7fb2a80b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2a81594a0 msgr2=0x7fb2a80b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2a81594a0 0x7fb2a80b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2a80c64c0 msgr2=0x7fb2a8152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2a80c64c0 0x7fb2a8152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 -- 172.21.15.16:0/1815820676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0094f0 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2a6ffd700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009290 tx=0x7fb29c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c026480 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c01f070 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c026850 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a815aee0 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.603+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a815b4a0 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.604+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a815bc60 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.605+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb29c01a430 con 0x7fb2a80cef40 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.605+0000 7fb28f7fe700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.605+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb29c021070 con 0x7fb2a80cef40 2023-10-06T11:42:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.605+0000 7fb2a77fe700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.606+0000 7fb2a77fe700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 0x7fb2940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb298003a20 tx=0x7fb298007f10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.610+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb29c053640 con 0x7fb2a80cef40 2023-10-06T11:42:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.803+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb2a815be80 con 0x7fb2a80cef40 2023-10-06T11:42:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.803+0000 7fb28f7fe700 1 -- 172.21.15.16:0/1815820676 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb29c056c60 con 0x7fb2a80cef40 2023-10-06T11:42:21.805 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.805+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 msgr2=0x7fb2940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.805+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 0x7fb2940693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb298003a20 tx=0x7fb298007f10 comp rx=0 tx=0).stop 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.805+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 msgr2=0x7fb2a80b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.805+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009290 tx=0x7fb29c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 shutdown_connections 2023-10-06T11:42:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb294066f30 0x7fb2940693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb2a81594a0 0x7fb2a80b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2a80c64c0 0x7fb2a8152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 --2- 172.21.15.16:0/1815820676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2a80cef40 0x7fb2a80b4fc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 >> 172.21.15.16:0/1815820676 conn(0x7fb2a8089f90 msgr2=0x7fb2a80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 shutdown_connections 2023-10-06T11:42:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.806+0000 7fb2acfef700 1 -- 172.21.15.16:0/1815820676 wait complete. 2023-10-06T11:42:21.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 Processor -- start 2023-10-06T11:42:21.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 -- start start 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9418159b10 0x7f9418164890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180cead0 con 0x7f94180cda10 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180b47b0 con 0x7f9418159740 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.941+0000 7f941e4af700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180affc0 con 0x7f9418159b10 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9418159b10 0x7f9418164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9418159b10 0x7f9418164890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57844/0 (socket says 172.21.15.16:57844) 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f94177fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94180cda10 0x7f94180b40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48468/0 (socket says 172.21.15.16:48468) 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 -- 172.21.15.16:0/1981562809 learned_addr learned my addr 172.21.15.16:0/1981562809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 -- 172.21.15.16:0/1981562809 >> v1:172.21.15.177:6789/0 conn(0x7f9418159740 legacy=0x7f94180b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 -- 172.21.15.16:0/1981562809 >> v1:172.21.15.16:6789/0 conn(0x7f94180cda10 legacy=0x7f94180b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 -- 172.21.15.16:0/1981562809 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180cdec0 con 0x7f9418159b10 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f9416ffd700 1 -- 172.21.15.16:0/1981562809 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152905895 0 0) 0x7f94180cead0 con 0x7f94180cda10 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f941ca4c700 1 --2- 172.21.15.16:0/1981562809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f9418164890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9410009f20 tx=0x7f9410009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e481f4f68e08542 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f9416ffd700 1 -- 172.21.15.16:0/1981562809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941000e070 con 0x7f9418159b10 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f9416ffd700 1 -- 172.21.15.16:0/1981562809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941000ba80 con 0x7f9418159b10 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.942+0000 7f9416ffd700 1 -- 172.21.15.16:0/1981562809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94100092e0 con 0x7f9418159b10 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 -- 172.21.15.16:0/1981562809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 msgr2=0x7f9418164890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 --2- 172.21.15.16:0/1981562809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f9418164890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9410009f20 tx=0x7f9410009840 comp rx=0 tx=0).stop 2023-10-06T11:42:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 -- 172.21.15.16:0/1981562809 shutdown_connections 2023-10-06T11:42:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 --2- 172.21.15.16:0/1981562809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f9418164890 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 -- 172.21.15.16:0/1981562809 >> 172.21.15.16:0/1981562809 conn(0x7f9418099a40 msgr2=0x7f941806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 -- 172.21.15.16:0/1981562809 shutdown_connections 2023-10-06T11:42:21.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.943+0000 7f941e4af700 1 -- 172.21.15.16:0/1981562809 wait complete. 2023-10-06T11:42:21.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.944+0000 7f941e4af700 1 Processor -- start 2023-10-06T11:42:21.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.944+0000 7f941e4af700 1 -- start start 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94180cda10 0x7f941816b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9418159740 0x7f941815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941816ac90 con 0x7f9418159b10 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941816b250 con 0x7f9418159740 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941e4af700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941809c5f0 con 0x7f94180cda10 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941ca4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941ca4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57850/0 (socket says 172.21.15.16:57850) 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f941ca4c700 1 -- 172.21.15.16:0/500212183 learned_addr learned my addr 172.21.15.16:0/500212183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f94177fe700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9418159740 0x7f941815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.945+0000 7f9417fff700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94180cda10 0x7f941816b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 -- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94180cda10 msgr2=0x7f941816b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94180cda10 0x7f941816b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 -- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9418159740 msgr2=0x7f941815c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9418159740 0x7f941815c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 -- 172.21.15.16:0/500212183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410009ae0 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941ca4c700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f941806d350 tx=0x7f9410018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410007cd0 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410007e70 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941809c8d0 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410026690 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.946+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941816bc20 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.947+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941816b3c0 con 0x7f9418159b10 2023-10-06T11:42:21.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.947+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f941001d420 con 0x7f9418159b10 2023-10-06T11:42:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.948+0000 7f93fffff700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.948+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f94100866f0 con 0x7f9418159b10 2023-10-06T11:42:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.948+0000 7f9417fff700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:21.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.948+0000 7f9417fff700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 0x7f94040693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9408003a80 tx=0x7f9408007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:21.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:21.952+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94100538b0 con 0x7f9418159b10 2023-10-06T11:42:22.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.147+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f941804c380 con 0x7f9418159b10 2023-10-06T11:42:22.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.147+0000 7f93fffff700 1 -- 172.21.15.16:0/500212183 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9410056ed0 con 0x7f9418159b10 2023-10-06T11:42:22.148 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 msgr2=0x7f94040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 0x7f94040693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9408003a80 tx=0x7f9408007ec0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 msgr2=0x7f941809b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f941806d350 tx=0x7f9410018000 comp rx=0 tx=0).stop 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 shutdown_connections 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9418159b10 0x7f941809b9a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9404066f30 0x7f94040693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f94180cda10 0x7f941816b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 --2- 172.21.15.16:0/500212183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9418159740 0x7f941815c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 >> 172.21.15.16:0/500212183 conn(0x7f9418099a40 msgr2=0x7f94180b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 shutdown_connections 2023-10-06T11:42:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.149+0000 7f941e4af700 1 -- 172.21.15.16:0/500212183 wait complete. 2023-10-06T11:42:22.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.284+0000 7f23a9fe0700 1 Processor -- start 2023-10-06T11:42:22.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.284+0000 7f23a9fe0700 1 -- start start 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a4159740 0x7f23a40b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f23a40cda10 0x7f23a40b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a40b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a40cead0 con 0x7f23a4159740 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a40affc0 con 0x7f23a40cda10 2023-10-06T11:42:22.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a9fe0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40cdec0 con 0x7f23a4159b10 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a4159740 0x7f23a40b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23a4159740 0x7f23a40b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57864/0 (socket says 172.21.15.16:57864) 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 -- 172.21.15.16:0/1937195206 learned_addr learned my addr 172.21.15.16:0/1937195206 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a2ffd700 1 --2- 172.21.15.16:0/1937195206 >> v2:172.21.15.177:3300/0 conn(0x7f23a40cda10 0x7f23a40b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a3fff700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a40b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 -- 172.21.15.16:0/1937195206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 msgr2=0x7f23a40b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a40b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 -- 172.21.15.16:0/1937195206 >> v2:172.21.15.177:3300/0 conn(0x7f23a40cda10 msgr2=0x7f23a40b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 --2- 172.21.15.16:0/1937195206 >> v2:172.21.15.177:3300/0 conn(0x7f23a40cda10 0x7f23a40b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 -- 172.21.15.16:0/1937195206 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a40ce1a0 con 0x7f23a4159740 2023-10-06T11:42:22.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a37fe700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4159740 0x7f23a40b3990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f23a406bd00 tx=0x7f2394009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da31d6c69353192e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.285+0000 7f23a27fc700 1 -- 172.21.15.16:0/1937195206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400b640 con 0x7f23a4159740 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a27fc700 1 -- 172.21.15.16:0/1937195206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239400b7a0 con 0x7f23a4159740 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a27fc700 1 -- 172.21.15.16:0/1937195206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239400bb60 con 0x7f23a4159740 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 -- 172.21.15.16:0/1937195206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4159740 msgr2=0x7f23a40b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4159740 0x7f23a40b3990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f23a406bd00 tx=0x7f2394009250 comp rx=0 tx=0).stop 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 -- 172.21.15.16:0/1937195206 shutdown_connections 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a40b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/1937195206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a4159740 0x7f23a40b3990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/1937195206 >> v2:172.21.15.177:3300/0 conn(0x7f23a40cda10 0x7f23a40b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 -- 172.21.15.16:0/1937195206 >> 172.21.15.16:0/1937195206 conn(0x7f23a4099a40 msgr2=0x7f23a406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 -- 172.21.15.16:0/1937195206 shutdown_connections 2023-10-06T11:42:22.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.286+0000 7f23a9fe0700 1 -- 172.21.15.16:0/1937195206 wait complete. 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.287+0000 7f23a9fe0700 1 Processor -- start 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.287+0000 7f23a9fe0700 1 -- start start 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4159740 0x7f23a409c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a409dd30 con 0x7f23a40cda10 2023-10-06T11:42:22.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a409d120 con 0x7f23a4159740 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a9fe0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a409d400 con 0x7f23a4159b10 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57870/0 (socket says 172.21.15.16:57870) 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 -- 172.21.15.16:0/586463019 learned_addr learned my addr 172.21.15.16:0/586463019 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a2ffd700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4159740 0x7f23a409c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a3fff700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 -- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 msgr2=0x7f23a415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 -- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4159740 msgr2=0x7f23a409c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4159740 0x7f23a409c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a37fe700 1 -- 172.21.15.16:0/586463019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940094f0 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.288+0000 7f23a3fff700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f23a37fe700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23a406d350 tx=0x7f2394017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394000d10 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239400bb20 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a409d6e0 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394011670 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.289+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a4157da0 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.290+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4157ff0 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.290+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23940119e0 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.290+0000 7f238b7fe700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 0x7f23900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.291+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2394085f60 con 0x7f23a40cda10 2023-10-06T11:42:22.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.291+0000 7f23a2ffd700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 0x7f23900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.291+0000 7f23a2ffd700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 0x7f23900693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ad070 tx=0x7f2398008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.295+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2394052fd0 con 0x7f23a40cda10 2023-10-06T11:42:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.489+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23a404c380 con 0x7f23a40cda10 2023-10-06T11:42:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.489+0000 7f238b7fe700 1 -- 172.21.15.16:0/586463019 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23940565f0 con 0x7f23a40cda10 2023-10-06T11:42:22.490 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 msgr2=0x7f23900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 0x7f23900693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ad070 tx=0x7f2398008040 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 msgr2=0x7f23a40af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23a406d350 tx=0x7f2394017000 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 shutdown_connections 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23a40cda10 0x7f23a40af1b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2390066f30 0x7f23900693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23a4159b10 0x7f23a415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 --2- 172.21.15.16:0/586463019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23a4159740 0x7f23a409c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 >> 172.21.15.16:0/586463019 conn(0x7f23a4099a40 msgr2=0x7f23a4158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 shutdown_connections 2023-10-06T11:42:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.491+0000 7f23a9fe0700 1 -- 172.21.15.16:0/586463019 wait complete. 2023-10-06T11:42:22.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.626+0000 7f617c870700 1 Processor -- start 2023-10-06T11:42:22.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.626+0000 7f617c870700 1 -- start start 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617c870700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61780cb0b0 0x7f61780cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617c870700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780ccbe0 con 0x7f617815bba0 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617c870700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780cc2d0 con 0x7f61780cd070 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617c870700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780ca160 con 0x7f61780cb0b0 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61780cb0b0 0x7f61780cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f6175d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f617815bba0 0x7f61780c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48482/0 (socket says 172.21.15.16:48482) 2023-10-06T11:42:22.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f617659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f61780cb0b0 0x7f61780cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57878/0 (socket says 172.21.15.16:57878) 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.627+0000 7f6175d9b700 1 -- 172.21.15.16:0/1440798621 learned_addr learned my addr 172.21.15.16:0/1440798621 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617659c700 1 -- 172.21.15.16:0/1440798621 >> v1:172.21.15.177:6789/0 conn(0x7f61780cd070 legacy=0x7f617815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617659c700 1 -- 172.21.15.16:0/1440798621 >> v1:172.21.15.16:6789/0 conn(0x7f617815bba0 legacy=0x7f61780c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617559a700 1 -- 172.21.15.16:0/1440798621 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719027200 0 0) 0x7f61780ccbe0 con 0x7f617815bba0 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617659c700 1 -- 172.21.15.16:0/1440798621 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780c9310 con 0x7f61780cb0b0 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617659c700 1 --2- 172.21.15.16:0/1440798621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cb0b0 0x7f61780cb480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61700098e0 tx=0x7f61700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b645790440b9a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617559a700 1 -- 172.21.15.16:0/1440798621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170009e80 con 0x7f61780cb0b0 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617559a700 1 -- 172.21.15.16:0/1440798621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617000b860 con 0x7f61780cb0b0 2023-10-06T11:42:22.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617559a700 1 -- 172.21.15.16:0/1440798621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617000bbe0 con 0x7f61780cb0b0 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617c870700 1 -- 172.21.15.16:0/1440798621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cb0b0 msgr2=0x7f61780cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.628+0000 7f617c870700 1 --2- 172.21.15.16:0/1440798621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cb0b0 0x7f61780cb480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61700098e0 tx=0x7f61700091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.629+0000 7f617c870700 1 -- 172.21.15.16:0/1440798621 shutdown_connections 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.629+0000 7f617c870700 1 --2- 172.21.15.16:0/1440798621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cb0b0 0x7f61780cb480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.629+0000 7f617c870700 1 -- 172.21.15.16:0/1440798621 >> 172.21.15.16:0/1440798621 conn(0x7f6178067c90 msgr2=0x7f6178068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.629+0000 7f617c870700 1 -- 172.21.15.16:0/1440798621 shutdown_connections 2023-10-06T11:42:22.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.629+0000 7f617c870700 1 -- 172.21.15.16:0/1440798621 wait complete. 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 Processor -- start 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 -- start start 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 0x7f6178154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 0x7f61780c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178169e00 con 0x7f61780cd070 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6178156cf0 con 0x7f617815bba0 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.630+0000 7f617c870700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6178156fd0 con 0x7f61780cb0b0 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57884/0 (socket says 172.21.15.16:57884) 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 -- 172.21.15.16:0/1466326210 learned_addr learned my addr 172.21.15.16:0/1466326210 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:22.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6176d9d700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 0x7f61780c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f617659c700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 0x7f6178154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 -- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 msgr2=0x7f6178154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 0x7f6178154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 -- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 msgr2=0x7f61780c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 0x7f61780c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 -- 172.21.15.16:0/1466326210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170009420 con 0x7f61780cd070 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6176d9d700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 0x7f61780c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f617659c700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 0x7f6178154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f6175d9b700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f616000c530 tx=0x7f6160007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160005b30 con 0x7f61780cd070 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160005c90 con 0x7f61780cd070 2023-10-06T11:42:22.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61781572b0 con 0x7f61780cd070 2023-10-06T11:42:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160007320 con 0x7f61780cd070 2023-10-06T11:42:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.631+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6178154860 con 0x7f61780cd070 2023-10-06T11:42:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.632+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617804c380 con 0x7f61780cd070 2023-10-06T11:42:22.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.633+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f616001b3b0 con 0x7f61780cd070 2023-10-06T11:42:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.633+0000 7f616e7fc700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 0x7f61580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.633+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6160083b70 con 0x7f61780cd070 2023-10-06T11:42:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.633+0000 7f617659c700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 0x7f61580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.634+0000 7f617659c700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 0x7f61580693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61700093f0 tx=0x7f6170017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.638+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6160050c80 con 0x7f61780cd070 2023-10-06T11:42:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.828+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61780cc590 con 0x7f61780cd070 2023-10-06T11:42:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.828+0000 7f616e7fc700 1 -- 172.21.15.16:0/1466326210 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61600170a0 con 0x7f61780cd070 2023-10-06T11:42:22.829 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 msgr2=0x7f61580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 0x7f61580693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61700093f0 tx=0x7f6170017000 comp rx=0 tx=0).stop 2023-10-06T11:42:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 msgr2=0x7f61780c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f616000c530 tx=0x7f6160007f90 comp rx=0 tx=0).stop 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 shutdown_connections 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6158066f30 0x7f61580693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f617815bba0 0x7f61780c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f61780cb0b0 0x7f6178154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 --2- 172.21.15.16:0/1466326210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61780cd070 0x7f61780c3dc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 >> 172.21.15.16:0/1466326210 conn(0x7f6178067c90 msgr2=0x7f6178155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 shutdown_connections 2023-10-06T11:42:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.830+0000 7f617c870700 1 -- 172.21.15.16:0/1466326210 wait complete. 2023-10-06T11:42:22.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 Processor -- start 2023-10-06T11:42:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 -- start start 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f16a40c65a0 0x7f16a40cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a40ccb20 con 0x7f16a40cd870 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a40cbf10 con 0x7f16a415cc20 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.961+0000 7f16a9694700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a40b3110 con 0x7f16a40c65a0 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a27fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f16a40cd870 0x7f16a40ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48492/0 (socket says 172.21.15.16:48492) 2023-10-06T11:42:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a27fc700 1 -- 172.21.15.16:0/1199843778 learned_addr learned my addr 172.21.15.16:0/1199843778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867603532 0 0) 0x7f16a40ccb20 con 0x7f16a40cd870 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a37fe700 1 --2- 172.21.15.16:0/1199843778 >> v2:172.21.15.177:3300/0 conn(0x7f16a40c65a0 0x7f16a40cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c003570 con 0x7f16a40cd870 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1749749917 0 0) 0x7f168c003570 con 0x7f16a40cd870 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 >> v2:172.21.15.177:3300/0 conn(0x7f16a40c65a0 msgr2=0x7f16a40cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 --2- 172.21.15.16:0/1199843778 >> v2:172.21.15.177:3300/0 conn(0x7f16a40c65a0 0x7f16a40cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 >> v1:172.21.15.177:6789/0 conn(0x7f16a415cc20 legacy=0x7f16a40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40cc1f0 con 0x7f16a40cd870 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f1698001470 con 0x7f16a40cd870 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 >> v1:172.21.15.16:6789/0 conn(0x7f16a40cd870 legacy=0x7f16a40ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c003a60 0x7f168c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 0x7f168c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f168c0052e0 0x7f168c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40b3110 con 0x7f168c004810 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00c2a0 con 0x7f168c003a60 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00c460 con 0x7f168c0052e0 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a27fc700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 0x7f168c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.962+0000 7f16a2ffd700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c003a60 0x7f168c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a37fe700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f168c0052e0 0x7f168c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 -- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f168c0052e0 msgr2=0x7f168c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f168c0052e0 0x7f168c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 -- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c003a60 msgr2=0x7f168c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c003a60 0x7f168c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 -- 172.21.15.16:0/1199843778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c003570 con 0x7f168c004810 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a37fe700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f168c0052e0 0x7f168c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a37fe700 1 -- 172.21.15.16:0/1199843778 reap_dead start 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a27fc700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 0x7f168c004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f169800c3c0 tx=0x7f1698001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51494dca0200345e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800cc90 con 0x7f168c004810 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800cdf0 con 0x7f168c004810 2023-10-06T11:42:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a1ffb700 1 -- 172.21.15.16:0/1199843778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698005b50 con 0x7f168c004810 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a9694700 1 -- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 msgr2=0x7f168c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.963+0000 7f16a9694700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 0x7f168c004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f169800c3c0 tx=0x7f1698001e50 comp rx=0 tx=0).stop 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.964+0000 7f16a9694700 1 -- 172.21.15.16:0/1199843778 shutdown_connections 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.964+0000 7f16a9694700 1 --2- 172.21.15.16:0/1199843778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f168c004810 0x7f168c004c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.964+0000 7f16a9694700 1 -- 172.21.15.16:0/1199843778 >> 172.21.15.16:0/1199843778 conn(0x7f16a4099b40 msgr2=0x7f16a406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.964+0000 7f16a9694700 1 -- 172.21.15.16:0/1199843778 shutdown_connections 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.964+0000 7f16a9694700 1 -- 172.21.15.16:0/1199843778 wait complete. 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.965+0000 7f16a9694700 1 Processor -- start 2023-10-06T11:42:22.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.965+0000 7f16a9694700 1 -- start start 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c004810 0x7f16a416af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a40c9130 0x7f16a40c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40ca220 con 0x7f16a4150170 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c9980 con 0x7f168c004810 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40c9c60 con 0x7f16a40c9130 2023-10-06T11:42:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57902/0 (socket says 172.21.15.16:57902) 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 -- 172.21.15.16:0/2036212389 learned_addr learned my addr 172.21.15.16:0/2036212389 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a2ffd700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c004810 0x7f16a416af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a27fc700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a40c9130 0x7f16a40c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 -- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a40c9130 msgr2=0x7f16a40c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a40c9130 0x7f16a40c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 -- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c004810 msgr2=0x7f16a416af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c004810 0x7f16a416af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 -- 172.21.15.16:0/2036212389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a40c9f40 con 0x7f16a4150170 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a37fe700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cbe20 tx=0x7f169c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c01b070 con 0x7f16a4150170 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c00f040 con 0x7f16a4150170 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a40b39e0 con 0x7f16a4150170 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00e050 con 0x7f16a4150170 2023-10-06T11:42:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.966+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a40c7d50 con 0x7f16a4150170 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.967+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a404c380 con 0x7f16a4150170 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.968+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f169c003020 con 0x7f16a4150170 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.968+0000 7f168affd700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 0x7f16900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.968+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f169c084b20 con 0x7f16a4150170 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.968+0000 7f16a2ffd700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 0x7f16900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.969+0000 7f16a2ffd700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 0x7f16900693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16a409e500 tx=0x7f1694007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:22.972+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f169c051c30 con 0x7f16a4150170 2023-10-06T11:42:23.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.165+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a40cc4d0 con 0x7f16a4150170 2023-10-06T11:42:23.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.166+0000 7f168affd700 1 -- 172.21.15.16:0/2036212389 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f169c055250 con 0x7f16a4150170 2023-10-06T11:42:23.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:23.168 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 msgr2=0x7f16900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 0x7f16900693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f16a409e500 tx=0x7f1694007d30 comp rx=0 tx=0).stop 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 msgr2=0x7f16a406e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f16a40cbe20 tx=0x7f169c00b040 comp rx=0 tx=0).stop 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 shutdown_connections 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16a4150170 0x7f16a406e5f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1690066f30 0x7f16900693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f168c004810 0x7f16a416af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 --2- 172.21.15.16:0/2036212389 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16a40c9130 0x7f16a40c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 >> 172.21.15.16:0/2036212389 conn(0x7f16a4099b40 msgr2=0x7f16a4150c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 shutdown_connections 2023-10-06T11:42:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.169+0000 7f16a9694700 1 -- 172.21.15.16:0/2036212389 wait complete. 2023-10-06T11:42:23.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:23.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.304+0000 7ff374382700 1 Processor -- start 2023-10-06T11:42:23.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.304+0000 7ff374382700 1 -- start start 2023-10-06T11:42:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff374382700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff36c0b09d0 0x7ff36c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff374382700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c0b4790 0x7ff36c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff374382700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36c0b1790 con 0x7ff36c159740 2023-10-06T11:42:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff374382700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36c0b0e60 con 0x7ff36c0b09d0 2023-10-06T11:42:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff374382700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0b1140 con 0x7ff36c0b4790 2023-10-06T11:42:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff36c0b09d0 0x7ff36c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37291f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c0b4790 0x7ff36c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff36c0b09d0 0x7ff36c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:35854/0 (socket says 172.21.15.16:35854) 2023-10-06T11:42:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 -- 172.21.15.16:0/3465375186 learned_addr learned my addr 172.21.15.16:0/3465375186 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 -- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c0b4790 msgr2=0x7ff36c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 --2- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c0b4790 0x7ff36c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37191d700 1 -- 172.21.15.16:0/3465375186 >> v1:172.21.15.177:6789/0 conn(0x7ff36c159740 legacy=0x7ff36c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff37191d700 1 -- 172.21.15.16:0/3465375186 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c0b1420 con 0x7ff36c0b09d0 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.305+0000 7ff37111c700 1 -- 172.21.15.16:0/3465375186 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970756552 0 0) 0x7ff36c0b1790 con 0x7ff36c159740 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff37191d700 1 --2- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b09d0 0x7ff36c159b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3600099b0 tx=0x7ff360009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adcc10662963aef2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff37111c700 1 -- 172.21.15.16:0/3465375186 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360009ea0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff37111c700 1 -- 172.21.15.16:0/3465375186 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36000b8f0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff37111c700 1 -- 172.21.15.16:0/3465375186 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000bc90 con 0x7ff36c0b09d0 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 -- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b09d0 msgr2=0x7ff36c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 --2- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b09d0 0x7ff36c159b10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff3600099b0 tx=0x7ff360009250 comp rx=0 tx=0).stop 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 -- 172.21.15.16:0/3465375186 shutdown_connections 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 --2- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c0b4790 0x7ff36c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 --2- 172.21.15.16:0/3465375186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b09d0 0x7ff36c159b10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.306+0000 7ff374382700 1 -- 172.21.15.16:0/3465375186 >> 172.21.15.16:0/3465375186 conn(0x7ff36c099a40 msgr2=0x7ff36c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.307+0000 7ff374382700 1 -- 172.21.15.16:0/3465375186 shutdown_connections 2023-10-06T11:42:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.307+0000 7ff374382700 1 -- 172.21.15.16:0/3465375186 wait complete. 2023-10-06T11:42:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 Processor -- start 2023-10-06T11:42:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 -- start start 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 0x7ff36c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c159740 0x7ff36c169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c09be30 con 0x7ff36c0b09d0 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0c63f0 con 0x7ff36c0b4790 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff374382700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c09b500 con 0x7ff36c159740 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff37211e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff37211e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57916/0 (socket says 172.21.15.16:57916) 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.308+0000 7ff37211e700 1 -- 172.21.15.16:0/245306420 learned_addr learned my addr 172.21.15.16:0/245306420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37291f700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c159740 0x7ff36c169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37191d700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 0x7ff36c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 -- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c159740 msgr2=0x7ff36c169b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c159740 0x7ff36c169b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 -- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 msgr2=0x7ff36c0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 0x7ff36c0c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 -- 172.21.15.16:0/245306420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3600094f0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37191d700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 0x7ff36c0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:23.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff37211e700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c530 tx=0x7ff36400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364011590 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3640116f0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364011ab0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c09b7e0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.309+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c16a0a0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.310+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c04c380 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.311+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff36401b3b0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.311+0000 7ff36a7fc700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 0x7ff3580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.311+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff364083c60 con 0x7ff36c0b09d0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.311+0000 7ff37191d700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 0x7ff3580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.312+0000 7ff37191d700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 0x7ff3580693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3600099b0 tx=0x7ff360017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.315+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff364050d70 con 0x7ff36c0b09d0 2023-10-06T11:42:23.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.506+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff36c0707b0 con 0x7ff36c0b09d0 2023-10-06T11:42:23.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.506+0000 7ff36a7fc700 1 -- 172.21.15.16:0/245306420 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff364017070 con 0x7ff36c0b09d0 2023-10-06T11:42:23.520 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 msgr2=0x7ff3580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 0x7ff3580693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3600099b0 tx=0x7ff360017000 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 msgr2=0x7ff36c0c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c530 tx=0x7ff36400c8f0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 shutdown_connections 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff36c0b09d0 0x7ff36c0c6770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff358066f30 0x7ff3580693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff36c159740 0x7ff36c169b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 --2- 172.21.15.16:0/245306420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff36c0b4790 0x7ff36c0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 >> 172.21.15.16:0/245306420 conn(0x7ff36c099a40 msgr2=0x7ff36c06db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 shutdown_connections 2023-10-06T11:42:23.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.508+0000 7ff374382700 1 -- 172.21.15.16:0/245306420 wait complete. 2023-10-06T11:42:23.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:23.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.644+0000 7fb71742a700 1 Processor -- start 2023-10-06T11:42:23.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.644+0000 7fb71742a700 1 -- start start 2023-10-06T11:42:23.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb710159740 0x7fb710159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 0x7fb7100b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb7100b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100b4a60 con 0x7fb710159740 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100b1790 con 0x7fb7100b09d0 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb71742a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100b0e60 con 0x7fb7100b4080 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb710159740 0x7fb710159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb710159740 0x7fb710159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57928/0 (socket says 172.21.15.16:57928) 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 -- 172.21.15.16:0/3391285472 learned_addr learned my addr 172.21.15.16:0/3391285472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 -- 172.21.15.16:0/3391285472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 msgr2=0x7fb7100b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 --2- 172.21.15.16:0/3391285472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb7100b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 -- 172.21.15.16:0/3391285472 >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 msgr2=0x7fb7100b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7149c5700 1 --2- 172.21.15.16:0/3391285472 >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 0x7fb7100b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 --2- 172.21.15.16:0/3391285472 >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 0x7fb7100b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 -- 172.21.15.16:0/3391285472 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100b1140 con 0x7fb710159740 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7149c5700 1 --2- 172.21.15.16:0/3391285472 >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 0x7fb7100b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb7151c6700 1 --2- 172.21.15.16:0/3391285472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb710159b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb710089030 tx=0x7fb70c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ec1e9091ff6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb707fff700 1 -- 172.21.15.16:0/3391285472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00b8f0 con 0x7fb710159740 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.645+0000 7fb707fff700 1 -- 172.21.15.16:0/3391285472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00ba50 con 0x7fb710159740 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb707fff700 1 -- 172.21.15.16:0/3391285472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00bdf0 con 0x7fb710159740 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 -- 172.21.15.16:0/3391285472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 msgr2=0x7fb710159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 --2- 172.21.15.16:0/3391285472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb710159b10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb710089030 tx=0x7fb70c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:23.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 -- 172.21.15.16:0/3391285472 shutdown_connections 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 --2- 172.21.15.16:0/3391285472 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb7100b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 --2- 172.21.15.16:0/3391285472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb710159b10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 --2- 172.21.15.16:0/3391285472 >> v2:172.21.15.177:3300/0 conn(0x7fb7100b09d0 0x7fb7100b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 -- 172.21.15.16:0/3391285472 >> 172.21.15.16:0/3391285472 conn(0x7fb710099a40 msgr2=0x7fb71006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 -- 172.21.15.16:0/3391285472 shutdown_connections 2023-10-06T11:42:23.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.646+0000 7fb71742a700 1 -- 172.21.15.16:0/3391285472 wait complete. 2023-10-06T11:42:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.647+0000 7fb71742a700 1 Processor -- start 2023-10-06T11:42:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.647+0000 7fb71742a700 1 -- start start 2023-10-06T11:42:23.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7100b09d0 0x7fb7100ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb71009cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c73f0 con 0x7fb710159740 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c7110 con 0x7fb7100b09d0 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb71742a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c7740 con 0x7fb7100b4080 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb7159c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb7159c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57942/0 (socket says 172.21.15.16:57942) 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb7159c7700 1 -- 172.21.15.16:0/2862648850 learned_addr learned my addr 172.21.15.16:0/2862648850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb7149c5700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb71009cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.648+0000 7fb7151c6700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7100b09d0 0x7fb7100ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 -- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 msgr2=0x7fb71009cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb71009cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 -- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7100b09d0 msgr2=0x7fb7100ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7100b09d0 0x7fb7100ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 -- 172.21.15.16:0/2862648850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0094f0 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7159c7700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb70800bc30 tx=0x7fb708009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080092e0 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70800c7a0 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7100c7a20 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800cb80 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.649+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100caa20 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.650+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7100c75a0 con 0x7fb710159740 2023-10-06T11:42:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.650+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb708012070 con 0x7fb710159740 2023-10-06T11:42:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.651+0000 7fb7057fa700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.651+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb7080843f0 con 0x7fb710159740 2023-10-06T11:42:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.651+0000 7fb7151c6700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:23.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.652+0000 7fb7151c6700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 0x7fb6f40693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb710089030 tx=0x7fb70c017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:23.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.655+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7100c75a0 con 0x7fb710159740 2023-10-06T11:42:23.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.847+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7100c75a0 con 0x7fb710159740 2023-10-06T11:42:23.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.848+0000 7fb7057fa700 1 -- 172.21.15.16:0/2862648850 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7100c75a0 con 0x7fb710159740 2023-10-06T11:42:23.874 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 msgr2=0x7fb6f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 0x7fb6f40693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb710089030 tx=0x7fb70c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:23.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 msgr2=0x7fb71009d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb70800bc30 tx=0x7fb708009840 comp rx=0 tx=0).stop 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 shutdown_connections 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb710159740 0x7fb71009d0d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6f4066f30 0x7fb6f40693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7100b09d0 0x7fb7100ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 --2- 172.21.15.16:0/2862648850 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7100b4080 0x7fb71009cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 >> 172.21.15.16:0/2862648850 conn(0x7fb710099a40 msgr2=0x7fb71006db30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 shutdown_connections 2023-10-06T11:42:23.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.850+0000 7fb71742a700 1 -- 172.21.15.16:0/2862648850 wait complete. 2023-10-06T11:42:23.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.997+0000 7f1da01d5700 1 Processor -- start 2023-10-06T11:42:24.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.997+0000 7f1da01d5700 1 -- start start 2023-10-06T11:42:24.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1da01d5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d980b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1da01d5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1d98159b10 0x7f1d980cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1da01d5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980cead0 con 0x7f1d980cda10 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1da01d5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d980affc0 con 0x7f1d98159b10 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1da01d5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980cdec0 con 0x7f1d98159740 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9d770700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d980cda10 0x7f1d980b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48502/0 (socket says 172.21.15.16:48502) 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9d770700 1 -- 172.21.15.16:0/615707288 learned_addr learned my addr 172.21.15.16:0/615707288 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9e772700 1 --2- 172.21.15.16:0/615707288 >> v2:172.21.15.177:3300/0 conn(0x7f1d98159b10 0x7f1d980cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9df71700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d980b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464445382 0 0) 0x7f1d980cead0 con 0x7f1d980cda10 2023-10-06T11:42:24.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d80003570 con 0x7f1d980cda10 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184070623 0 0) 0x7f1d80003570 con 0x7f1d980cda10 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 msgr2=0x7f1d980b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d980b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 >> v2:172.21.15.177:3300/0 conn(0x7f1d98159b10 msgr2=0x7f1d980cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 --2- 172.21.15.16:0/615707288 >> v2:172.21.15.177:3300/0 conn(0x7f1d98159b10 0x7f1d980cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980ce1a0 con 0x7f1d980cda10 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f1d8c001470 con 0x7f1d980cda10 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 >> v1:172.21.15.16:6789/0 conn(0x7f1d980cda10 legacy=0x7f1d980b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80003a60 0x7f1d80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d80004810 0x7f1d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.998+0000 7f1d9cf6f700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980affc0 con 0x7f1d80004810 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980cdec0 con 0x7f1d80003a60 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8000ba60 con 0x7f1d98159740 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d80004810 0x7f1d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9e772700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9df71700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80003a60 0x7f1d80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 -- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 msgr2=0x7f1d800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 -- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80003a60 msgr2=0x7f1d80003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80003a60 0x7f1d80003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 -- 172.21.15.16:0/615707288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80003570 con 0x7f1d80004810 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9e772700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d98159740 0x7f1d800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:24.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9d770700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d80004810 0x7f1d80004c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c3f0 tx=0x7f1d8c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d6a169f8a4a337 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00cc80 con 0x7f1d80004810 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00ce00 con 0x7f1d80004810 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:23.999+0000 7f1d9cf6f700 1 -- 172.21.15.16:0/615707288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c005b70 con 0x7f1d80004810 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 -- 172.21.15.16:0/615707288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d80004810 msgr2=0x7f1d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 --2- 172.21.15.16:0/615707288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d80004810 0x7f1d80004c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00c3f0 tx=0x7f1d8c001e90 comp rx=0 tx=0).stop 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1d9e772700 1 -- 172.21.15.16:0/615707288 reap_dead start 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 -- 172.21.15.16:0/615707288 shutdown_connections 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 -- 172.21.15.16:0/615707288 >> 172.21.15.16:0/615707288 conn(0x7f1d98099a40 msgr2=0x7f1d9806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 -- 172.21.15.16:0/615707288 shutdown_connections 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.000+0000 7f1da01d5700 1 -- 172.21.15.16:0/615707288 wait complete. 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.001+0000 7f1da01d5700 1 Processor -- start 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.001+0000 7f1da01d5700 1 -- start start 2023-10-06T11:42:24.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 0x7f1d9815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d9815cf50 0x7f1d980ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d98159fe0 con 0x7f1d981565f0 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980ad650 con 0x7f1d80004810 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1da01d5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980ad930 con 0x7f1d9815cf50 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57964/0 (socket says 172.21.15.16:57964) 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 -- 172.21.15.16:0/2991423854 learned_addr learned my addr 172.21.15.16:0/2991423854 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9df71700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 0x7f1d9815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9d770700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d9815cf50 0x7f1d980ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 -- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d9815cf50 msgr2=0x7f1d980ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d9815cf50 0x7f1d980ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 -- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 msgr2=0x7f1d9815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 0x7f1d9815c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 -- 172.21.15.16:0/2991423854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980adc10 con 0x7f1d981565f0 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9df71700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 0x7f1d9815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d9e772700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b3240 tx=0x7f1d8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8801b070 con 0x7f1d981565f0 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800f040 con 0x7f1d981565f0 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.002+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800e050 con 0x7f1d981565f0 2023-10-06T11:42:24.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.003+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d980adef0 con 0x7f1d981565f0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.003+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d98156fb0 con 0x7f1d981565f0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.004+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9815a150 con 0x7f1d981565f0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.004+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d88003020 con 0x7f1d981565f0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.004+0000 7f1d95ffb700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.005+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1d88084bb0 con 0x7f1d981565f0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.005+0000 7f1d9df71700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.005+0000 7f1d9df71700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 0x7f1d840693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d90006dc0 tx=0x7f1d90008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.009+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d88051cc0 con 0x7f1d981565f0 2023-10-06T11:42:24.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.200+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d980ae0e0 con 0x7f1d981565f0 2023-10-06T11:42:24.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.200+0000 7f1d95ffb700 1 -- 172.21.15.16:0/2991423854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d880552e0 con 0x7f1d981565f0 2023-10-06T11:42:24.201 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:24.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 msgr2=0x7f1d840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 0x7f1d840693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1d90006dc0 tx=0x7f1d90008040 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 msgr2=0x7f1d98156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b3240 tx=0x7f1d8800b040 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 shutdown_connections 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d981565f0 0x7f1d98156a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1d84066f30 0x7f1d840693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d80004810 0x7f1d9815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 --2- 172.21.15.16:0/2991423854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1d9815cf50 0x7f1d980ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 >> 172.21.15.16:0/2991423854 conn(0x7f1d98099a40 msgr2=0x7f1d9806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 shutdown_connections 2023-10-06T11:42:24.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.202+0000 7f1da01d5700 1 -- 172.21.15.16:0/2991423854 wait complete. 2023-10-06T11:42:24.215 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.336+0000 7fcc7d12d700 1 Processor -- start 2023-10-06T11:42:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.336+0000 7fcc7d12d700 1 -- start start 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7d12d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc78159270 0x7fcc7806acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7d12d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7d12d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7806a820 con 0x7fcc78159640 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7d12d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc78070b00 con 0x7fcc78159270 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7d12d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7806f330 con 0x7fcc7806e9b0 2023-10-06T11:42:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc77fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc78159270 0x7fcc7806acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc7c92c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc77fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcc78159270 0x7fcc7806acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:35920/0 (socket says 172.21.15.16:35920) 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc77fff700 1 -- 172.21.15.16:0/3092884109 learned_addr learned my addr 172.21.15.16:0/3092884109 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.337+0000 7fcc777fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fcc78159640 0x7fcc780703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:50432/0 (socket says 172.21.15.16:50432) 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc77fff700 1 -- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 msgr2=0x7fcc7806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc77fff700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc77fff700 1 -- 172.21.15.16:0/3092884109 >> v1:172.21.15.177:6789/0 conn(0x7fcc78159640 legacy=0x7fcc780703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc77fff700 1 -- 172.21.15.16:0/3092884109 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780abe40 con 0x7fcc78159270 2023-10-06T11:42:24.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc7c92c700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc77fff700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7806acb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc680099b0 tx=0x7fcc68009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339ebabca571283c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc76ffd700 1 -- 172.21.15.16:0/3092884109 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68009ea0 con 0x7fcc78159270 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc76ffd700 1 -- 172.21.15.16:0/3092884109 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6800b860 con 0x7fcc78159270 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc76ffd700 1 -- 172.21.15.16:0/3092884109 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6800bc20 con 0x7fcc78159270 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 msgr2=0x7fcc7806acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.338+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7806acb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc680099b0 tx=0x7fcc68009250 comp rx=0 tx=0).stop 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3092884109 shutdown_connections 2023-10-06T11:42:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3092884109 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7806acb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3092884109 >> 172.21.15.16:0/3092884109 conn(0x7fcc78067cb0 msgr2=0x7fcc780680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3092884109 shutdown_connections 2023-10-06T11:42:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.339+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3092884109 wait complete. 2023-10-06T11:42:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.340+0000 7fcc7d12d700 1 Processor -- start 2023-10-06T11:42:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.340+0000 7fcc7d12d700 1 -- start start 2023-10-06T11:42:24.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7815d320 con 0x7fcc78159640 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78152de0 con 0x7fcc78159270 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78153080 con 0x7fcc7806e9b0 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57976/0 (socket says 172.21.15.16:57976) 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 -- 172.21.15.16:0/3014220972 learned_addr learned my addr 172.21.15.16:0/3014220972 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc77fff700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc777fe700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 -- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 msgr2=0x7fcc7814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 -- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 msgr2=0x7fcc7815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 -- 172.21.15.16:0/3014220972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc70009a20 con 0x7fcc78159640 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7c92c700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009f20 tx=0x7fcc7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7000cc80 con 0x7fcc78159640 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7000f040 con 0x7fcc78159640 2023-10-06T11:42:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.341+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680094f0 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.342+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70014480 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.342+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7815cfb0 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.342+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc781535a0 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.343+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc70003020 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.343+0000 7fcc5ffff700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 0x7fcc640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.343+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcc70085090 con 0x7fcc78159640 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.343+0000 7fcc77fff700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 0x7fcc640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.344+0000 7fcc77fff700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 0x7fcc640693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc680099b0 tx=0x7fcc68017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.348+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc700521a0 con 0x7fcc78159640 2023-10-06T11:42:24.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.540+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc7815d460 con 0x7fcc78159640 2023-10-06T11:42:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.541+0000 7fcc5ffff700 1 -- 172.21.15.16:0/3014220972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc700557c0 con 0x7fcc78159640 2023-10-06T11:42:24.562 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 msgr2=0x7fcc640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 0x7fcc640693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc680099b0 tx=0x7fcc68017000 comp rx=0 tx=0).stop 2023-10-06T11:42:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 msgr2=0x7fcc78156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009f20 tx=0x7fcc7000b040 comp rx=0 tx=0).stop 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 shutdown_connections 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc78159640 0x7fcc78156af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.542+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcc64066f30 0x7fcc640693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.543+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc7806e9b0 0x7fcc7814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.543+0000 7fcc7d12d700 1 --2- 172.21.15.16:0/3014220972 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc78159270 0x7fcc7815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.543+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 >> 172.21.15.16:0/3014220972 conn(0x7fcc78067cb0 msgr2=0x7fcc780aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.543+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 shutdown_connections 2023-10-06T11:42:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.543+0000 7fcc7d12d700 1 -- 172.21.15.16:0/3014220972 wait complete. 2023-10-06T11:42:24.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.684+0000 7f1dd2cf2700 1 Processor -- start 2023-10-06T11:42:24.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.684+0000 7f1dd2cf2700 1 -- start start 2023-10-06T11:42:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd2cf2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd2cf2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1dcc159790 0x7f1dcc15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd2cf2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc15a650 con 0x7f1dcc0b53f0 2023-10-06T11:42:24.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd2cf2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc0b5f60 con 0x7f1dcc159790 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd2cf2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc0b61c0 con 0x7f1dcc1593c0 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd14ef700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1dcc159790 0x7f1dcc15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd14ef700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1dcc159790 0x7f1dcc15d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:35952/0 (socket says 172.21.15.16:35952) 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd14ef700 1 -- 172.21.15.16:0/1101611521 learned_addr learned my addr 172.21.15.16:0/1101611521 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.685+0000 7f1dd1cf0700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd14ef700 1 -- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 msgr2=0x7f1dcc15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:24.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd14ef700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd14ef700 1 -- 172.21.15.16:0/1101611521 >> v1:172.21.15.177:6789/0 conn(0x7f1dcc0b53f0 legacy=0x7f1dcc0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd14ef700 1 -- 172.21.15.16:0/1101611521 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc159d20 con 0x7f1dcc159790 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd14ef700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc159790 0x7f1dcc15d750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009990 tx=0x7f1dc0009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6571e6bb9c005500 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd0cee700 1 -- 172.21.15.16:0/1101611521 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0009e80 con 0x7f1dcc159790 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd0cee700 1 -- 172.21.15.16:0/1101611521 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0005c10 con 0x7f1dcc159790 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.686+0000 7f1dd0cee700 1 -- 172.21.15.16:0/1101611521 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc000b650 con 0x7f1dcc159790 2023-10-06T11:42:24.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc159790 msgr2=0x7f1dcc15d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc159790 0x7f1dcc15d750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009990 tx=0x7f1dc0009220 comp rx=0 tx=0).stop 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1101611521 shutdown_connections 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1101611521 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc159790 0x7f1dcc15d750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1101611521 >> 172.21.15.16:0/1101611521 conn(0x7f1dcc08a020 msgr2=0x7f1dcc099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1101611521 shutdown_connections 2023-10-06T11:42:24.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.687+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1101611521 wait complete. 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.688+0000 7f1dd2cf2700 1 Processor -- start 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.688+0000 7f1dd2cf2700 1 -- start start 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 0x7f1dcc158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc14e0e0 con 0x7f1dcc159790 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc14e340 con 0x7f1dcc0b53f0 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd2cf2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc14e600 con 0x7f1dcc1593c0 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 0x7f1dcc158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd14ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd1cf0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd1cf0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:35964/0 (socket says 172.21.15.16:35964) 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd14ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:59424/0 (socket says 172.21.15.16:59424) 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd1cf0700 1 -- 172.21.15.16:0/1509492865 learned_addr learned my addr 172.21.15.16:0/1509492865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 -- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 msgr2=0x7f1dcc157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 -- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 msgr2=0x7f1dcc14bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 -- 172.21.15.16:0/1509492865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009480 con 0x7f1dcc159790 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd1cf0700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd14ef700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:24.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.689+0000 7f1dd24f1700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 0x7f1dcc158440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400c830 tx=0x7f1dc4001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.690+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4012630 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.690+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc4012790 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.690+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4012b50 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.690+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc14e8e0 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.690+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc14c160 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.691+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc14ee00 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.691+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dc4011380 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.691+0000 7f1dc9ffb700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 0x7f1db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.692+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1dc40847d0 con 0x7f1dcc159790 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.692+0000 7f1dd1cf0700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 0x7f1db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:24.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.692+0000 7f1dd1cf0700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 0x7f1db80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc011520 tx=0x7f1dbc009120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:24.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.696+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dc40518e0 con 0x7f1dcc159790 2023-10-06T11:42:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.891+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1dcc04c380 con 0x7f1dcc159790 2023-10-06T11:42:24.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.892+0000 7f1dc9ffb700 1 -- 172.21.15.16:0/1509492865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f1dc4054f00 con 0x7f1dcc159790 2023-10-06T11:42:24.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:24.893 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:24.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 msgr2=0x7f1db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 0x7f1db80693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc011520 tx=0x7f1dbc009120 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 msgr2=0x7f1dcc158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 0x7f1dcc158440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400c830 tx=0x7f1dc4001690 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 shutdown_connections 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc159790 0x7f1dcc158440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f1db8066f30 0x7f1db80693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dcc0b53f0 0x7f1dcc14bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 --2- 172.21.15.16:0/1509492865 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dcc1593c0 0x7f1dcc157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 >> 172.21.15.16:0/1509492865 conn(0x7f1dcc08a020 msgr2=0x7f1dcc14d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:24.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 shutdown_connections 2023-10-06T11:42:24.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:24.894+0000 7f1dd2cf2700 1 -- 172.21.15.16:0/1509492865 wait complete. 2023-10-06T11:42:24.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:24.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.029+0000 7f378da39700 1 Processor -- start 2023-10-06T11:42:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.029+0000 7f378da39700 1 -- start start 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.029+0000 7f378da39700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f378815cef0 0x7f37880cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f378da39700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880cb700 0x7f37880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f378da39700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37880b4e90 con 0x7f378815cb20 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f378da39700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37880cbb60 con 0x7f378815cef0 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f378da39700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880b4280 con 0x7f37880cb700 2023-10-06T11:42:25.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3786ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f378815cb20 0x7f37880b5320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48518/0 (socket says 172.21.15.16:48518) 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f378815cef0 0x7f37880cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3786ffd700 1 -- 172.21.15.16:0/2504863703 learned_addr learned my addr 172.21.15.16:0/2504863703 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37877fe700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880cb700 0x7f37880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3785ffb700 1 -- 172.21.15.16:0/2504863703 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155121081 0 0) 0x7f37880b4e90 con 0x7f378815cb20 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3785ffb700 1 -- 172.21.15.16:0/2504863703 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3770003570 con 0x7f378815cb20 2023-10-06T11:42:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 -- 172.21.15.16:0/2504863703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880cb700 msgr2=0x7f37880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880cb700 0x7f37880b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 -- 172.21.15.16:0/2504863703 >> v1:172.21.15.16:6789/0 conn(0x7f378815cb20 legacy=0x7f37880b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 -- 172.21.15.16:0/2504863703 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880b4560 con 0x7f378815cef0 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f37867fc700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378815cef0 0x7f37880cb1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f377c0098e0 tx=0x7f377c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2626297d8e43e135 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3785ffb700 1 -- 172.21.15.16:0/2504863703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c011630 con 0x7f378815cef0 2023-10-06T11:42:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3785ffb700 1 -- 172.21.15.16:0/2504863703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c011790 con 0x7f378815cef0 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.030+0000 7f3785ffb700 1 -- 172.21.15.16:0/2504863703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377c011b30 con 0x7f378815cef0 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 -- 172.21.15.16:0/2504863703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378815cef0 msgr2=0x7f37880cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378815cef0 0x7f37880cb1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f377c0098e0 tx=0x7f377c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 -- 172.21.15.16:0/2504863703 shutdown_connections 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37880cb700 0x7f37880b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 --2- 172.21.15.16:0/2504863703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f378815cef0 0x7f37880cb1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 -- 172.21.15.16:0/2504863703 >> 172.21.15.16:0/2504863703 conn(0x7f378806d8f0 msgr2=0x7f378806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 -- 172.21.15.16:0/2504863703 shutdown_connections 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.031+0000 7f378da39700 1 -- 172.21.15.16:0/2504863703 wait complete. 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.032+0000 7f378da39700 1 Processor -- start 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.032+0000 7f378da39700 1 -- start start 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f378815cb20 0x7f37880a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378815cef0 0x7f37880c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880b5e20 con 0x7f37880cb700 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880a8b80 con 0x7f378815cb20 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f378da39700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37880a7d10 con 0x7f378815cef0 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57998/0 (socket says 172.21.15.16:57998) 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 -- 172.21.15.16:0/1822673255 learned_addr learned my addr 172.21.15.16:0/1822673255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f37867fc700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f378815cb20 0x7f37880a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f37877fe700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378815cef0 0x7f37880c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 -- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378815cef0 msgr2=0x7f37880c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378815cef0 0x7f37880c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 -- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f378815cb20 msgr2=0x7f37880a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.033+0000 7f3786ffd700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f378815cb20 0x7f37880a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f3786ffd700 1 -- 172.21.15.16:0/1822673255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c009420 con 0x7f37880cb700 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f3786ffd700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f377800c790 tx=0x7f3778007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780072e0 con 0x7f37880cb700 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3778007440 con 0x7f37880cb700 2023-10-06T11:42:25.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880a9010 con 0x7f37880cb700 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37780057f0 con 0x7f37880cb700 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.034+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37880a8250 con 0x7f37880cb700 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.035+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37880b2e80 con 0x7f37880cb700 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.035+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3778005be0 con 0x7f37880cb700 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.036+0000 7f376effd700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 0x7f37740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.036+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3778083ab0 con 0x7f37880cb700 2023-10-06T11:42:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.036+0000 7f37867fc700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 0x7f37740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.036+0000 7f37867fc700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 0x7f37740693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f378815d350 tx=0x7f377c017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.040+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3778050c70 con 0x7f37880cb700 2023-10-06T11:42:25.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.233+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f37880b4840 con 0x7f37880cb700 2023-10-06T11:42:25.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.233+0000 7f376effd700 1 -- 172.21.15.16:0/1822673255 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3778017070 con 0x7f37880cb700 2023-10-06T11:42:25.304 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 msgr2=0x7f37740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 0x7f37740693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f378815d350 tx=0x7f377c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 msgr2=0x7f37880b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f377800c790 tx=0x7f3778007c80 comp rx=0 tx=0).stop 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 shutdown_connections 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37880cb700 0x7f37880b58e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3774066f30 0x7f37740693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f378815cef0 0x7f37880c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 --2- 172.21.15.16:0/1822673255 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f378815cb20 0x7f37880a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 >> 172.21.15.16:0/1822673255 conn(0x7f378806d8f0 msgr2=0x7f37880cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 shutdown_connections 2023-10-06T11:42:25.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.235+0000 7f378da39700 1 -- 172.21.15.16:0/1822673255 wait complete. 2023-10-06T11:42:25.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.427+0000 7fb3ede3d700 1 Processor -- start 2023-10-06T11:42:25.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.427+0000 7fb3ede3d700 1 -- start start 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ede3d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb3e80cd670 0x7fb3e81641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ede3d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80cffc0 con 0x7fb3e81594a0 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ede3d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e80cf3b0 con 0x7fb3e80cef40 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ede3d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e80cf690 con 0x7fb3e80cd670 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ece3b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb3e81594a0 0x7fb3e80ce830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48534/0 (socket says 172.21.15.16:48534) 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ece3b700 1 -- 172.21.15.16:0/1047811053 learned_addr learned my addr 172.21.15.16:0/1047811053 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3ed63c700 1 --2- 172.21.15.16:0/1047811053 >> v2:172.21.15.177:3300/0 conn(0x7fb3e80cd670 0x7fb3e81641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417214938 0 0) 0x7fb3e80cffc0 con 0x7fb3e81594a0 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d0003570 con 0x7fb3e81594a0 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.428+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611092482 0 0) 0x7fb3d0003570 con 0x7fb3e81594a0 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ed63c700 1 -- 172.21.15.16:0/1047811053 >> v1:172.21.15.177:6789/0 conn(0x7fb3e80cef40 legacy=0x7fb3e80ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ed63c700 1 -- 172.21.15.16:0/1047811053 >> v1:172.21.15.16:6789/0 conn(0x7fb3e81594a0 legacy=0x7fb3e80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ed63c700 1 -- 172.21.15.16:0/1047811053 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e80cf970 con 0x7fb3e80cd670 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ed63c700 1 --2- 172.21.15.16:0/1047811053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e80cd670 0x7fb3e81641d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009e60 tx=0x7fb3e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d957e3baccfa1c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e000e070 con 0x7fb3e80cd670 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e000b850 con 0x7fb3e80cd670 2023-10-06T11:42:25.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3e77fe700 1 -- 172.21.15.16:0/1047811053 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80015f0 con 0x7fb3e80cd670 2023-10-06T11:42:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1047811053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e80cd670 msgr2=0x7fb3e81641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1047811053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e80cd670 0x7fb3e81641d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009e60 tx=0x7fb3e0009780 comp rx=0 tx=0).stop 2023-10-06T11:42:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1047811053 shutdown_connections 2023-10-06T11:42:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1047811053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e80cd670 0x7fb3e81641d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1047811053 >> 172.21.15.16:0/1047811053 conn(0x7fb3e8089f90 msgr2=0x7fb3e80998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1047811053 shutdown_connections 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.429+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1047811053 wait complete. 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 Processor -- start 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 -- start start 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3e80cd670 0x7fb3e8150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 0x7fb3e80c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80c79e0 con 0x7fb3e80cef40 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e815b4d0 con 0x7fb3e81594a0 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.431+0000 7fb3ede3d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80c76b0 con 0x7fb3e80cd670 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58010/0 (socket says 172.21.15.16:58010) 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 -- 172.21.15.16:0/1601010945 learned_addr learned my addr 172.21.15.16:0/1601010945 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3ed63c700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 0x7fb3e80c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3ece3b700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3e80cd670 0x7fb3e8150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 -- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3e80cd670 msgr2=0x7fb3e8150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3e80cd670 0x7fb3e8150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 -- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 msgr2=0x7fb3e80c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 0x7fb3e80c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 -- 172.21.15.16:0/1601010945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e0009a20 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3ed63c700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 0x7fb3e80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e7fff700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c810 tx=0x7fb3dc007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc012630 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc012790 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e815b750 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc012b50 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.432+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e815bd10 con 0x7fb3e80cef40 2023-10-06T11:42:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.433+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e8089530 con 0x7fb3e80cef40 2023-10-06T11:42:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.434+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3dc01b400 con 0x7fb3e80cef40 2023-10-06T11:42:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.434+0000 7fb3e4ff9700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 0x7fb3d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.434+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb3dc084400 con 0x7fb3e80cef40 2023-10-06T11:42:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.434+0000 7fb3ece3b700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 0x7fb3d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.435+0000 7fb3ece3b700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 0x7fb3d40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8001380 tx=0x7fb3d8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.438+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3dc051510 con 0x7fb3e80cef40 2023-10-06T11:42:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.632+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3e80c7b20 con 0x7fb3e80cef40 2023-10-06T11:42:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.632+0000 7fb3e4ff9700 1 -- 172.21.15.16:0/1601010945 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3dc054b30 con 0x7fb3e80cef40 2023-10-06T11:42:25.634 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.634+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 msgr2=0x7fb3d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.634+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 0x7fb3d40693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8001380 tx=0x7fb3d8009040 comp rx=0 tx=0).stop 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.634+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 msgr2=0x7fb3e815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.634+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc00c810 tx=0x7fb3dc007ed0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 shutdown_connections 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3d4066f30 0x7fb3d40693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb3e81594a0 0x7fb3e80c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3e80cd670 0x7fb3e8150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 --2- 172.21.15.16:0/1601010945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3e80cef40 0x7fb3e815af90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 >> 172.21.15.16:0/1601010945 conn(0x7fb3e8089f90 msgr2=0x7fb3e80b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 shutdown_connections 2023-10-06T11:42:25.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.635+0000 7fb3ede3d700 1 -- 172.21.15.16:0/1601010945 wait complete. 2023-10-06T11:42:25.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.768+0000 7fc3ee5e2700 1 Processor -- start 2023-10-06T11:42:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3ee5e2700 1 -- start start 2023-10-06T11:42:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3ee5e2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3ee5e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e813c9f0 con 0x7fc3e80cb290 2023-10-06T11:42:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3ee5e2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e806dff0 con 0x7fc3e80cc7d0 2023-10-06T11:42:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3ee5e2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e815e270 con 0x7fc3e80cb660 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3e7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3e80cb290 0x7fc3e80cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48540/0 (socket says 172.21.15.16:48540) 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.769+0000 7fc3e7fff700 1 -- 172.21.15.16:0/757630749 learned_addr learned my addr 172.21.15.16:0/757630749 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3ecb7f700 1 --2- 172.21.15.16:0/757630749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493088988 0 0) 0x7fc3d8001110 con 0x7fc3e80cb290 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d4003570 con 0x7fc3e80cb290 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310496033 0 0) 0x7fc3e806dff0 con 0x7fc3e80cc7d0 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3ecb7f700 1 -- 172.21.15.16:0/757630749 >> v1:172.21.15.177:6789/0 conn(0x7fc3e80cc7d0 legacy=0x7fc3e815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3ecb7f700 1 -- 172.21.15.16:0/757630749 >> v1:172.21.15.16:6789/0 conn(0x7fc3e80cb290 legacy=0x7fc3e80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3ecb7f700 1 -- 172.21.15.16:0/757630749 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e815ae30 con 0x7fc3e80cb660 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3ecb7f700 1 --2- 172.21.15.16:0/757630749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e815dc10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009e90 tx=0x7fc3e0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2b80045318bf14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000e070 con 0x7fc3e80cb660 2023-10-06T11:42:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000bc30 con 0x7fc3e80cb660 2023-10-06T11:42:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.770+0000 7fc3e6ffd700 1 -- 172.21.15.16:0/757630749 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0012430 con 0x7fc3e80cb660 2023-10-06T11:42:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/757630749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 msgr2=0x7fc3e815dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/757630749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e815dc10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009e90 tx=0x7fc3e0009780 comp rx=0 tx=0).stop 2023-10-06T11:42:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/757630749 shutdown_connections 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/757630749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e815dc10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/757630749 >> 172.21.15.16:0/757630749 conn(0x7fc3e8067c90 msgr2=0x7fc3e8068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/757630749 shutdown_connections 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.771+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/757630749 wait complete. 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.772+0000 7fc3ee5e2700 1 Processor -- start 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.772+0000 7fc3ee5e2700 1 -- start start 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3e80cc7d0 0x7fc3e8169a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8156b70 con 0x7fc3e80cb290 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80b1290 con 0x7fc3e80cc7d0 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ee5e2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e81588d0 con 0x7fc3e80cb660 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58012/0 (socket says 172.21.15.16:58012) 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 -- 172.21.15.16:0/1341178352 learned_addr learned my addr 172.21.15.16:0/1341178352 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e77fe700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3ecb7f700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3e80cc7d0 0x7fc3e8169a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 -- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 msgr2=0x7fc3e8156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e8156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 -- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3e80cc7d0 msgr2=0x7fc3e8169a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3e80cc7d0 0x7fc3e8169a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.773+0000 7fc3e7fff700 1 -- 172.21.15.16:0/1341178352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0009a20 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3e7fff700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800c7c0 tx=0x7fc3d8007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8005b30 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d8005c90 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e816ab20 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8007300 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.774+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e8158b70 con 0x7fc3e80cb290 2023-10-06T11:42:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.775+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e804c380 con 0x7fc3e80cb290 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.775+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3d80076b0 con 0x7fc3e80cb290 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.775+0000 7fc3dffff700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 0x7fc3c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.776+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc3d8083970 con 0x7fc3e80cb290 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.776+0000 7fc3e77fe700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 0x7fc3c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.776+0000 7fc3e77fe700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 0x7fc3c80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3e815a460 tx=0x7fc3d0007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.780+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3d8050a80 con 0x7fc3e80cb290 2023-10-06T11:42:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.972+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3e813c9f0 con 0x7fc3e80cb290 2023-10-06T11:42:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.972+0000 7fc3dffff700 1 -- 172.21.15.16:0/1341178352 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3e813c9f0 con 0x7fc3e80cb290 2023-10-06T11:42:25.973 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 msgr2=0x7fc3c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 0x7fc3c80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3e815a460 tx=0x7fc3d0007ec0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 msgr2=0x7fc3e81560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800c7c0 tx=0x7fc3d8007c30 comp rx=0 tx=0).stop 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 shutdown_connections 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e80cb290 0x7fc3e81560f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c8066f30 0x7fc3c80693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3e80cc7d0 0x7fc3e8169a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 --2- 172.21.15.16:0/1341178352 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3e80cb660 0x7fc3e8156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 >> 172.21.15.16:0/1341178352 conn(0x7fc3e8067c90 msgr2=0x7fc3e815be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 shutdown_connections 2023-10-06T11:42:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:25.974+0000 7fc3ee5e2700 1 -- 172.21.15.16:0/1341178352 wait complete. 2023-10-06T11:42:25.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.108+0000 7f0cf9b65700 1 Processor -- start 2023-10-06T11:42:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.108+0000 7f0cf9b65700 1 -- start start 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.108+0000 7f0cf9b65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0cf4159740 0x7f0cf40b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf9b65700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40cead0 con 0x7f0cf40cda10 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf9b65700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf40affc0 con 0x7f0cf4159b10 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf9b65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf40cdec0 con 0x7f0cf4159740 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0cf4159740 0x7f0cf40b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0cf40cda10 0x7f0cf40b3ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48542/0 (socket says 172.21.15.16:48542) 2023-10-06T11:42:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf2ffd700 1 -- 172.21.15.16:0/77668960 learned_addr learned my addr 172.21.15.16:0/77668960 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf37fe700 1 -- 172.21.15.16:0/77668960 >> v1:172.21.15.177:6789/0 conn(0x7f0cf4159b10 legacy=0x7f0cf40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf37fe700 1 -- 172.21.15.16:0/77668960 >> v1:172.21.15.16:6789/0 conn(0x7f0cf40cda10 legacy=0x7f0cf40b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf37fe700 1 -- 172.21.15.16:0/77668960 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf40ce1a0 con 0x7f0cf4159740 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf27fc700 1 -- 172.21.15.16:0/77668960 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749213668 0 0) 0x7f0cf40cead0 con 0x7f0cf40cda10 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf37fe700 1 --2- 172.21.15.16:0/77668960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159740 0x7f0cf40b3990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009e60 tx=0x7f0ce4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d946929e6b70ecad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf27fc700 1 -- 172.21.15.16:0/77668960 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400e070 con 0x7f0cf4159740 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf27fc700 1 -- 172.21.15.16:0/77668960 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400bc30 con 0x7f0cf4159740 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.109+0000 7f0cf27fc700 1 -- 172.21.15.16:0/77668960 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4012430 con 0x7f0cf4159740 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 -- 172.21.15.16:0/77668960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159740 msgr2=0x7f0cf40b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/77668960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159740 0x7f0cf40b3990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009e60 tx=0x7f0ce4009780 comp rx=0 tx=0).stop 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 -- 172.21.15.16:0/77668960 shutdown_connections 2023-10-06T11:42:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/77668960 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159740 0x7f0cf40b3990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 -- 172.21.15.16:0/77668960 >> 172.21.15.16:0/77668960 conn(0x7f0cf4099a40 msgr2=0x7f0cf406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 -- 172.21.15.16:0/77668960 shutdown_connections 2023-10-06T11:42:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.110+0000 7f0cf9b65700 1 -- 172.21.15.16:0/77668960 wait complete. 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.111+0000 7f0cf9b65700 1 Processor -- start 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.111+0000 7f0cf9b65700 1 -- start start 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cf40cda10 0x7f0cf415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf4159740 0x7f0cf4155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415b120 con 0x7f0cf4159b10 2023-10-06T11:42:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4155850 con 0x7f0cf4159740 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf9b65700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf415ab60 con 0x7f0cf40cda10 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58030/0 (socket says 172.21.15.16:58030) 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 -- 172.21.15.16:0/3747155493 learned_addr learned my addr 172.21.15.16:0/3747155493 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf37fe700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cf40cda10 0x7f0cf415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf2ffd700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf4159740 0x7f0cf4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 -- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cf40cda10 msgr2=0x7f0cf415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cf40cda10 0x7f0cf415c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 -- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf4159740 msgr2=0x7f0cf4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf4159740 0x7f0cf4155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.112+0000 7f0cf3fff700 1 -- 172.21.15.16:0/3747155493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4009a20 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0cf3fff700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b4e0 tx=0x7f0cec006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00be90 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec00f510 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf415ae40 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00f8f0 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.113+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4155cd0 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.114+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf409a3b0 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.114+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cec00fd20 con 0x7f0cf4159b10 2023-10-06T11:42:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.115+0000 7f0ceb7fe700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 0x7f0cd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.115+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0cec083f30 con 0x7f0cf4159b10 2023-10-06T11:42:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.115+0000 7f0cf37fe700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 0x7f0cd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.115+0000 7f0cf37fe700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 0x7f0cd40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009e60 tx=0x7f0ce4018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.119+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cf409a3b0 con 0x7f0cf4159b10 2023-10-06T11:42:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.310+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0cf409a3b0 con 0x7f0cf4159b10 2023-10-06T11:42:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.311+0000 7f0ceb7fe700 1 -- 172.21.15.16:0/3747155493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0cf409a3b0 con 0x7f0cf4159b10 2023-10-06T11:42:26.312 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 msgr2=0x7f0cd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 0x7f0cd40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009e60 tx=0x7f0ce4018000 comp rx=0 tx=0).stop 2023-10-06T11:42:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 msgr2=0x7f0cf415b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00b4e0 tx=0x7f0cec006f90 comp rx=0 tx=0).stop 2023-10-06T11:42:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 shutdown_connections 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cf4159b10 0x7f0cf415b3c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0cd4066f30 0x7f0cd40693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0cf40cda10 0x7f0cf415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 --2- 172.21.15.16:0/3747155493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0cf4159740 0x7f0cf4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 >> 172.21.15.16:0/3747155493 conn(0x7f0cf4099a40 msgr2=0x7f0cf4156450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 shutdown_connections 2023-10-06T11:42:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.313+0000 7f0cf9b65700 1 -- 172.21.15.16:0/3747155493 wait complete. 2023-10-06T11:42:26.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:26.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.444+0000 7fd5e4b44700 1 Processor -- start 2023-10-06T11:42:26.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.444+0000 7fd5e4b44700 1 -- start start 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd5e01595b0 0x7fd5e00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5e0159980 0x7fd5e00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 0x7fd5e00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cbf20 con 0x7fd5e0159980 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cc200 con 0x7fd5e01595b0 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5e4b44700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00cc4e0 con 0x7fd5e00cec70 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5ddd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5e0159980 0x7fd5e00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5ddd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5e0159980 0x7fd5e00cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58034/0 (socket says 172.21.15.16:58034) 2023-10-06T11:42:26.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5ddd9b700 1 -- 172.21.15.16:0/3131044237 learned_addr learned my addr 172.21.15.16:0/3131044237 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5ded9d700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 0x7fd5e00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.445+0000 7fd5de59c700 1 --2- 172.21.15.16:0/3131044237 >> v2:172.21.15.177:3300/0 conn(0x7fd5e01595b0 0x7fd5e00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 -- 172.21.15.16:0/3131044237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 msgr2=0x7fd5e00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 0x7fd5e00cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 -- 172.21.15.16:0/3131044237 >> v2:172.21.15.177:3300/0 conn(0x7fd5e01595b0 msgr2=0x7fd5e00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 --2- 172.21.15.16:0/3131044237 >> v2:172.21.15.177:3300/0 conn(0x7fd5e01595b0 0x7fd5e00cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 -- 172.21.15.16:0/3131044237 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00cc7c0 con 0x7fd5e0159980 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ded9d700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 0x7fd5e00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5ddd9b700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e00cd680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0089790 tx=0x7fd5c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829b61ba43eebf47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5dd59a700 1 -- 172.21.15.16:0/3131044237 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800b860 con 0x7fd5e0159980 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5dd59a700 1 -- 172.21.15.16:0/3131044237 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c800b9c0 con 0x7fd5e0159980 2023-10-06T11:42:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5dd59a700 1 -- 172.21.15.16:0/3131044237 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800bd60 con 0x7fd5e0159980 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5e4b44700 1 -- 172.21.15.16:0/3131044237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 msgr2=0x7fd5e00cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e00cd680 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0089790 tx=0x7fd5c8009250 comp rx=0 tx=0).stop 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.446+0000 7fd5e4b44700 1 -- 172.21.15.16:0/3131044237 shutdown_connections 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e00cec70 0x7fd5e00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/3131044237 >> v2:172.21.15.177:3300/0 conn(0x7fd5e01595b0 0x7fd5e00cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/3131044237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e00cd680 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 -- 172.21.15.16:0/3131044237 >> 172.21.15.16:0/3131044237 conn(0x7fd5e0099b40 msgr2=0x7fd5e0087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 -- 172.21.15.16:0/3131044237 shutdown_connections 2023-10-06T11:42:26.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.447+0000 7fd5e4b44700 1 -- 172.21.15.16:0/3131044237 wait complete. 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 Processor -- start 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 -- start start 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e00cec70 0x7fd5e00b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e01595b0 0x7fd5e013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e014f2c0 con 0x7fd5e0159980 2023-10-06T11:42:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e014f5a0 con 0x7fd5e00cec70 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5e4b44700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e014f880 con 0x7fd5e01595b0 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.448+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58046/0 (socket says 172.21.15.16:58046) 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 -- 172.21.15.16:0/2611446900 learned_addr learned my addr 172.21.15.16:0/2611446900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5de59c700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e00cec70 0x7fd5e00b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ddd9b700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e01595b0 0x7fd5e013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 -- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e01595b0 msgr2=0x7fd5e013fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e01595b0 0x7fd5e013fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 -- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e00cec70 msgr2=0x7fd5e00b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e00cec70 0x7fd5e00b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 -- 172.21.15.16:0/2611446900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80094f0 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5ded9d700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000d180 tx=0x7fd5d0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d000dcc0 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d000f040 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e014fb60 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0015690 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.449+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e00b5f10 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.450+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00b6160 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.451+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5d001e3b0 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.451+0000 7fd5d67fc700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.451+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd5d0084b00 con 0x7fd5e0159980 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.451+0000 7fd5de59c700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.452+0000 7fd5de59c700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 0x7fd5c00693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8006fd0 tx=0x7fd5d8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.455+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5d0051c10 con 0x7fd5e0159980 2023-10-06T11:42:26.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.653+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5e004c380 con 0x7fd5e0159980 2023-10-06T11:42:26.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.654+0000 7fd5d67fc700 1 -- 172.21.15.16:0/2611446900 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fd5d001a090 con 0x7fd5e0159980 2023-10-06T11:42:26.655 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:26.655 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:26.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 msgr2=0x7fd5c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 0x7fd5c00693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8006fd0 tx=0x7fd5d8008040 comp rx=0 tx=0).stop 2023-10-06T11:42:26.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 msgr2=0x7fd5e01402b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000d180 tx=0x7fd5d0009a50 comp rx=0 tx=0).stop 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 shutdown_connections 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5e0159980 0x7fd5e01402b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd5c0066f30 0x7fd5c00693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5e00cec70 0x7fd5e00b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 --2- 172.21.15.16:0/2611446900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd5e01595b0 0x7fd5e013fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 >> 172.21.15.16:0/2611446900 conn(0x7fd5e0099b40 msgr2=0x7fd5e006f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 shutdown_connections 2023-10-06T11:42:26.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.656+0000 7fd5e4b44700 1 -- 172.21.15.16:0/2611446900 wait complete. 2023-10-06T11:42:26.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:26.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.791+0000 7f574310f700 1 Processor -- start 2023-10-06T11:42:26.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.791+0000 7f574310f700 1 -- start start 2023-10-06T11:42:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574310f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574310f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f573c0cd670 0x7f573c1631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574310f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0cffc0 con 0x7f573c0ced70 2023-10-06T11:42:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574310f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0cf3b0 con 0x7f573c0cd670 2023-10-06T11:42:26.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574310f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0cf690 con 0x7f573c1594a0 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574210d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574290e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f573c0cd670 0x7f573c1631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574210d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:59526/0 (socket says 172.21.15.16:59526) 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574210d700 1 -- 172.21.15.16:0/1868879547 learned_addr learned my addr 172.21.15.16:0/1868879547 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.792+0000 7f574190c700 1 --1- 172.21.15.16:0/1868879547 >> v1:172.21.15.177:6789/0 conn(0x7f573c0ced70 0x7f573c0ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:50462/0 (socket says 172.21.15.16:50462) 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574290e700 1 -- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 msgr2=0x7f573c0ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574290e700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574290e700 1 -- 172.21.15.16:0/1868879547 >> v1:172.21.15.177:6789/0 conn(0x7f573c0ced70 legacy=0x7f573c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574290e700 1 -- 172.21.15.16:0/1868879547 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0cf970 con 0x7f573c0cd670 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574210d700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574290e700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0cd670 0x7f573c1631c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f572c009e60 tx=0x7f572c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edec4688ae680324 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574110b700 1 -- 172.21.15.16:0/1868879547 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00e070 con 0x7f573c0cd670 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574110b700 1 -- 172.21.15.16:0/1868879547 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00bb60 con 0x7f573c0cd670 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.793+0000 7f574110b700 1 -- 172.21.15.16:0/1868879547 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c012430 con 0x7f573c0cd670 2023-10-06T11:42:26.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 -- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0cd670 msgr2=0x7f573c1631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0cd670 0x7f573c1631c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f572c009e60 tx=0x7f572c009780 comp rx=0 tx=0).stop 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 -- 172.21.15.16:0/1868879547 shutdown_connections 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0cd670 0x7f573c1631c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 --2- 172.21.15.16:0/1868879547 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c1594a0 0x7f573c0ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 -- 172.21.15.16:0/1868879547 >> 172.21.15.16:0/1868879547 conn(0x7f573c089f90 msgr2=0x7f573c0998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 -- 172.21.15.16:0/1868879547 shutdown_connections 2023-10-06T11:42:26.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.794+0000 7f574310f700 1 -- 172.21.15.16:0/1868879547 wait complete. 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.795+0000 7f574310f700 1 Processor -- start 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.795+0000 7f574310f700 1 -- start start 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c0cd670 0x7f573c147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c06da20 con 0x7f573c1594a0 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c06dd00 con 0x7f573c0ced70 2023-10-06T11:42:26.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574310f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c155480 con 0x7f573c0cd670 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574190c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58052/0 (socket says 172.21.15.16:58052) 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574210d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c0cd670 0x7f573c147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574190c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:36066/0 (socket says 172.21.15.16:36066) 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 -- 172.21.15.16:0/1452060236 learned_addr learned my addr 172.21.15.16:0/1452060236 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 -- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c0cd670 msgr2=0x7f573c147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c0cd670 0x7f573c147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 -- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 msgr2=0x7f573c0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 -- 172.21.15.16:0/1452060236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734009480 con 0x7f573c1594a0 2023-10-06T11:42:26.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574190c700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.796+0000 7f574290e700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f572c013000 tx=0x7f572c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c027470 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c0275f0 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c009a20 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c0279b0 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c0c8460 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.797+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c06de70 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.798+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f572c024070 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.798+0000 7f573a7fc700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.798+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f572c087060 con 0x7f573c1594a0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.799+0000 7f574210d700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:26.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.799+0000 7f574210d700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 0x7f57280693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5734009f30 tx=0x7f5734009220 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:26.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.803+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f572c054170 con 0x7f573c1594a0 2023-10-06T11:42:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.996+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f573c06ce00 con 0x7f573c1594a0 2023-10-06T11:42:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.996+0000 7f573a7fc700 1 -- 172.21.15.16:0/1452060236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f572c057790 con 0x7f573c1594a0 2023-10-06T11:42:26.999 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 msgr2=0x7f57280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 0x7f57280693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5734009f30 tx=0x7f5734009220 comp rx=0 tx=0).stop 2023-10-06T11:42:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 msgr2=0x7f573c0c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f572c013000 tx=0x7f572c019000 comp rx=0 tx=0).stop 2023-10-06T11:42:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 shutdown_connections 2023-10-06T11:42:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573c1594a0 0x7f573c0c7b50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5728066f30 0x7f57280693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f573c0cd670 0x7f573c147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 --2- 172.21.15.16:0/1452060236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f573c0ced70 0x7f573c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 >> 172.21.15.16:0/1452060236 conn(0x7f573c089f90 msgr2=0x7f573c15f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 shutdown_connections 2023-10-06T11:42:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:26.998+0000 7f574310f700 1 -- 172.21.15.16:0/1452060236 wait complete. 2023-10-06T11:42:27.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.133+0000 7f36489ad700 1 Processor -- start 2023-10-06T11:42:27.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.133+0000 7f36489ad700 1 -- start start 2023-10-06T11:42:27.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f36489ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f36489ad700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36440b09d0 0x7f36440b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f36489ad700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36440b1790 con 0x7f36440b4080 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f36489ad700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440b0e60 con 0x7f36440b09d0 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f36489ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36440b1140 con 0x7f3644159740 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f3641d9b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f36440b09d0 0x7f36440b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:59550/0 (socket says 172.21.15.16:59550) 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 -- 172.21.15.16:0/2054066136 learned_addr learned my addr 172.21.15.16:0/2054066136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f3642d9d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f36440b4080 0x7f36440aa820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:50464/0 (socket says 172.21.15.16:50464) 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 -- 172.21.15.16:0/2054066136 >> v2:172.21.15.177:3300/0 conn(0x7f36440b09d0 msgr2=0x7f36440b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 --2- 172.21.15.16:0/2054066136 >> v2:172.21.15.177:3300/0 conn(0x7f36440b09d0 0x7f36440b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 -- 172.21.15.16:0/2054066136 >> v1:172.21.15.177:6789/0 conn(0x7f36440b4080 legacy=0x7f36440aa820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.134+0000 7f364259c700 1 -- 172.21.15.16:0/2054066136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440b1420 con 0x7f3644159740 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f364259c700 1 --2- 172.21.15.16:0/2054066136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36340099b0 tx=0x7f3634009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=523298b926c82b63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f364159a700 1 -- 172.21.15.16:0/2054066136 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634009ea0 con 0x7f3644159740 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f364159a700 1 -- 172.21.15.16:0/2054066136 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363400b8f0 con 0x7f3644159740 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f364159a700 1 -- 172.21.15.16:0/2054066136 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363400bc90 con 0x7f3644159740 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 -- 172.21.15.16:0/2054066136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 msgr2=0x7f3644159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 --2- 172.21.15.16:0/2054066136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36340099b0 tx=0x7f3634009250 comp rx=0 tx=0).stop 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 -- 172.21.15.16:0/2054066136 shutdown_connections 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 --2- 172.21.15.16:0/2054066136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3644159740 0x7f3644159b10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 --2- 172.21.15.16:0/2054066136 >> v2:172.21.15.177:3300/0 conn(0x7f36440b09d0 0x7f36440b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 -- 172.21.15.16:0/2054066136 >> 172.21.15.16:0/2054066136 conn(0x7f3644099a40 msgr2=0x7f364406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 -- 172.21.15.16:0/2054066136 shutdown_connections 2023-10-06T11:42:27.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.135+0000 7f36489ad700 1 -- 172.21.15.16:0/2054066136 wait complete. 2023-10-06T11:42:27.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.136+0000 7f36489ad700 1 Processor -- start 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 -- start start 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b09d0 0x7f3644158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 0x7f3644157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364416c120 con 0x7f3644159740 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364409db80 con 0x7f36440b4080 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f36489ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364409deb0 con 0x7f36440b09d0 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f3642d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58066/0 (socket says 172.21.15.16:58066) 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.137+0000 7f3642d9d700 1 -- 172.21.15.16:0/2167322992 learned_addr learned my addr 172.21.15.16:0/2167322992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f364259c700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b09d0 0x7f3644158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3641d9b700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 0x7f3644157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 -- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b09d0 msgr2=0x7f3644158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b09d0 0x7f3644158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 -- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 msgr2=0x7f3644157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 0x7f3644157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 -- 172.21.15.16:0/2167322992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340094f0 con 0x7f3644159740 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3641d9b700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 0x7f3644157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f3642d9d700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f363c00cd00 tx=0x7f363c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00ce60 con 0x7f3644159740 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c005df0 con 0x7f3644159740 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364409d5c0 con 0x7f3644159740 2023-10-06T11:42:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00abb0 con 0x7f3644159740 2023-10-06T11:42:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.138+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f364409e190 con 0x7f3644159740 2023-10-06T11:42:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.139+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364416c2d0 con 0x7f3644159740 2023-10-06T11:42:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.139+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f363c0070f0 con 0x7f3644159740 2023-10-06T11:42:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.140+0000 7f36327fc700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 0x7f36240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.140+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f363c0847a0 con 0x7f3644159740 2023-10-06T11:42:27.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.140+0000 7f364259c700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 0x7f36240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.141+0000 7f364259c700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 0x7f36240693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36340099b0 tx=0x7f3634017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.144+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f364416c2d0 con 0x7f3644159740 2023-10-06T11:42:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.336+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f364404c380 con 0x7f3644159740 2023-10-06T11:42:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.336+0000 7f36327fc700 1 -- 172.21.15.16:0/2167322992 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f363c051930 con 0x7f3644159740 2023-10-06T11:42:27.338 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.338+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 msgr2=0x7f36240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.338+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 0x7f36240693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36340099b0 tx=0x7f3634017000 comp rx=0 tx=0).stop 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.338+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 msgr2=0x7f36440c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.338+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f363c00cd00 tx=0x7f363c001b10 comp rx=0 tx=0).stop 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 shutdown_connections 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3644159740 0x7f36440c54a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3624066f30 0x7f36240693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f36440b09d0 0x7f3644158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 --2- 172.21.15.16:0/2167322992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36440b4080 0x7f3644157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 >> 172.21.15.16:0/2167322992 conn(0x7f3644099a40 msgr2=0x7f36440b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 shutdown_connections 2023-10-06T11:42:27.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.339+0000 7f36489ad700 1 -- 172.21.15.16:0/2167322992 wait complete. 2023-10-06T11:42:27.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.473+0000 7fdb99a4a700 1 Processor -- start 2023-10-06T11:42:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.473+0000 7fdb99a4a700 1 -- start start 2023-10-06T11:42:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb94143100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb9406e530 0x7fdb940cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb940cc600 0x7fdb940caef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb9415e210 con 0x7fdb9406e530 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb940cca60 con 0x7fdb940cc600 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb99a4a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9406d520 con 0x7fdb94142d30 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb9406e530 0x7fdb940cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb9406e530 0x7fdb940cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58078/0 (socket says 172.21.15.16:58078) 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 -- 172.21.15.16:0/457154742 learned_addr learned my addr 172.21.15.16:0/457154742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 -- 172.21.15.16:0/457154742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 msgr2=0x7fdb94143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 --2- 172.21.15.16:0/457154742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb94143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 -- 172.21.15.16:0/457154742 >> v2:172.21.15.177:3300/0 conn(0x7fdb940cc600 msgr2=0x7fdb940caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 --2- 172.21.15.16:0/457154742 >> v2:172.21.15.177:3300/0 conn(0x7fdb940cc600 0x7fdb940caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 -- 172.21.15.16:0/457154742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9406e0a0 con 0x7fdb9406e530 2023-10-06T11:42:27.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.474+0000 7fdb927fc700 1 --2- 172.21.15.16:0/457154742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9406e530 0x7fdb940cc0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406a7e0 tx=0x7fdb7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362d085db44d400b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb91ffb700 1 -- 172.21.15.16:0/457154742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00b860 con 0x7fdb9406e530 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb91ffb700 1 -- 172.21.15.16:0/457154742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00b9c0 con 0x7fdb9406e530 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb91ffb700 1 -- 172.21.15.16:0/457154742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00bd60 con 0x7fdb9406e530 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 -- 172.21.15.16:0/457154742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9406e530 msgr2=0x7fdb940cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/457154742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9406e530 0x7fdb940cc0c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdb9406a7e0 tx=0x7fdb7c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 -- 172.21.15.16:0/457154742 shutdown_connections 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/457154742 >> v2:172.21.15.177:3300/0 conn(0x7fdb940cc600 0x7fdb940caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/457154742 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb94143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/457154742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb9406e530 0x7fdb940cc0c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 -- 172.21.15.16:0/457154742 >> 172.21.15.16:0/457154742 conn(0x7fdb9406cee0 msgr2=0x7fdb94067cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 -- 172.21.15.16:0/457154742 shutdown_connections 2023-10-06T11:42:27.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.475+0000 7fdb99a4a700 1 -- 172.21.15.16:0/457154742 wait complete. 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.476+0000 7fdb99a4a700 1 Processor -- start 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 -- start start 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 0x7fdb940b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb9416a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940b1f50 con 0x7fdb940cc600 2023-10-06T11:42:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9416abf0 con 0x7fdb9406e530 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb99a4a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9416aed0 con 0x7fdb94142d30 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb927fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58082/0 (socket says 172.21.15.16:58082) 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.477+0000 7fdb927fc700 1 -- 172.21.15.16:0/3410773889 learned_addr learned my addr 172.21.15.16:0/3410773889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb937fe700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb9416a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb92ffd700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 0x7fdb940b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 -- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 msgr2=0x7fdb9416a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb9416a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 -- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 msgr2=0x7fdb940b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 0x7fdb940b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 -- 172.21.15.16:0/3410773889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0094f0 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb92ffd700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 0x7fdb940b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb937fe700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb9416a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb927fc700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009290 tx=0x7fdb7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00bd50 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c01a660 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01aa40 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9416b1b0 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.478+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9416a870 con 0x7fdb940cc600 2023-10-06T11:42:27.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.479+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9404c380 con 0x7fdb940cc600 2023-10-06T11:42:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.479+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb7c01b430 con 0x7fdb940cc600 2023-10-06T11:42:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.480+0000 7fdb8affd700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.480+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdb7c085e70 con 0x7fdb940cc600 2023-10-06T11:42:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.480+0000 7fdb92ffd700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.481+0000 7fdb92ffd700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 0x7fdb740693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb9413dc20 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.484+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb7c052f80 con 0x7fdb940cc600 2023-10-06T11:42:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.681+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb940b2090 con 0x7fdb940cc600 2023-10-06T11:42:27.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.681+0000 7fdb8affd700 1 -- 172.21.15.16:0/3410773889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb7c02a0a0 con 0x7fdb940cc600 2023-10-06T11:42:27.682 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:27.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 msgr2=0x7fdb740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 0x7fdb740693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdb9413dc20 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2023-10-06T11:42:27.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 msgr2=0x7fdb941582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009290 tx=0x7fdb7c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 shutdown_connections 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb74066f30 0x7fdb740693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb94142d30 0x7fdb9416a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb9406e530 0x7fdb940b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 --2- 172.21.15.16:0/3410773889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb940cc600 0x7fdb941582c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 >> 172.21.15.16:0/3410773889 conn(0x7fdb9406cee0 msgr2=0x7fdb941591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 shutdown_connections 2023-10-06T11:42:27.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.683+0000 7fdb99a4a700 1 -- 172.21.15.16:0/3410773889 wait complete. 2023-10-06T11:42:27.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.816+0000 7fe118396700 1 Processor -- start 2023-10-06T11:42:27.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.816+0000 7fe118396700 1 -- start start 2023-10-06T11:42:27.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe118396700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe110159740 0x7fe1100b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe118396700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 0x7fe1100b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe118396700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1100cead0 con 0x7fe110159b10 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe118396700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1100affc0 con 0x7fe110159740 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe118396700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1100cdec0 con 0x7fe1100cda10 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe110159740 0x7fe1100b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe110159740 0x7fe1100b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58096/0 (socket says 172.21.15.16:58096) 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 -- 172.21.15.16:0/3211713916 learned_addr learned my addr 172.21.15.16:0/3211713916 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe115931700 1 --2- 172.21.15.16:0/3211713916 >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 0x7fe1100b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 -- 172.21.15.16:0/3211713916 >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 msgr2=0x7fe1100b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 --2- 172.21.15.16:0/3211713916 >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 0x7fe1100b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 -- 172.21.15.16:0/3211713916 >> v1:172.21.15.177:6789/0 conn(0x7fe110159b10 legacy=0x7fe11015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.817+0000 7fe116132700 1 -- 172.21.15.16:0/3211713916 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100ce1a0 con 0x7fe110159740 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe115931700 1 --2- 172.21.15.16:0/3211713916 >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 0x7fe1100b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe116132700 1 --2- 172.21.15.16:0/3211713916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe1100b3990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bd00 tx=0x7fe10c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505e163360901e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe115130700 1 -- 172.21.15.16:0/3211713916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00b820 con 0x7fe110159740 2023-10-06T11:42:27.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe115130700 1 -- 172.21.15.16:0/3211713916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c00b980 con 0x7fe110159740 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe115130700 1 -- 172.21.15.16:0/3211713916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c00bd20 con 0x7fe110159740 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 -- 172.21.15.16:0/3211713916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 msgr2=0x7fe1100b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 --2- 172.21.15.16:0/3211713916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe1100b3990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bd00 tx=0x7fe10c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 -- 172.21.15.16:0/3211713916 shutdown_connections 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 --2- 172.21.15.16:0/3211713916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe1100b3990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 --2- 172.21.15.16:0/3211713916 >> v2:172.21.15.177:3300/0 conn(0x7fe1100cda10 0x7fe1100b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 -- 172.21.15.16:0/3211713916 >> 172.21.15.16:0/3211713916 conn(0x7fe110099a40 msgr2=0x7fe11006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 -- 172.21.15.16:0/3211713916 shutdown_connections 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.818+0000 7fe118396700 1 -- 172.21.15.16:0/3211713916 wait complete. 2023-10-06T11:42:27.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.819+0000 7fe118396700 1 Processor -- start 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 -- start start 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 0x7fe11014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe110159b10 0x7fe11009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11009bb60 con 0x7fe110159740 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015af30 con 0x7fe1100cda10 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe118396700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11009d2c0 con 0x7fe110159b10 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe115931700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe115931700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58102/0 (socket says 172.21.15.16:58102) 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe115931700 1 -- 172.21.15.16:0/687206026 learned_addr learned my addr 172.21.15.16:0/687206026 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.820+0000 7fe116132700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 0x7fe11014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe116933700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe110159b10 0x7fe11009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 -- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe110159b10 msgr2=0x7fe11009b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe110159b10 0x7fe11009b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 -- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 msgr2=0x7fe11014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 0x7fe11014aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 -- 172.21.15.16:0/687206026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c0094f0 con 0x7fe110159740 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe116132700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 0x7fe11014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:27.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe115931700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c830 tx=0x7fe100008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100011600 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100011760 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11009d5a0 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100011b20 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.821+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11009dbe0 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.822+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11014b0d0 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.822+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe10000a3e0 con 0x7fe110159740 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.823+0000 7fe1067fc700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 0x7fe0f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:27.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.823+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe100084b50 con 0x7fe110159740 2023-10-06T11:42:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.823+0000 7fe116132700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 0x7fe0f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.823+0000 7fe116132700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 0x7fe0f40693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bd00 tx=0x7fe10c016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:27.827+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe11014b0d0 con 0x7fe110159740 2023-10-06T11:42:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.021+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe11014b0d0 con 0x7fe110159740 2023-10-06T11:42:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.022+0000 7fe1067fc700 1 -- 172.21.15.16:0/687206026 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe11014b0d0 con 0x7fe110159740 2023-10-06T11:42:28.023 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 msgr2=0x7fe0f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 0x7fe0f40693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe11006bd00 tx=0x7fe10c016000 comp rx=0 tx=0).stop 2023-10-06T11:42:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 msgr2=0x7fe11009af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c830 tx=0x7fe100008040 comp rx=0 tx=0).stop 2023-10-06T11:42:28.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 shutdown_connections 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe0f4066f30 0x7fe0f40693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe110159b10 0x7fe11009b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1100cda10 0x7fe11014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 --2- 172.21.15.16:0/687206026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe110159740 0x7fe11009af80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 >> 172.21.15.16:0/687206026 conn(0x7fe110099a40 msgr2=0x7fe110157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 shutdown_connections 2023-10-06T11:42:28.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.024+0000 7fe118396700 1 -- 172.21.15.16:0/687206026 wait complete. 2023-10-06T11:42:28.118 INFO:tasks.cephfs_test_runner:Starting test: test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) 2023-10-06T11:42:28.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-10-06T11:42:28.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.237+0000 7f4a6411d700 1 Processor -- start 2023-10-06T11:42:28.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.238+0000 7f4a6411d700 1 -- start start 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.238+0000 7f4a6411d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a5c0cd870 0x7f4a5c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.238+0000 7f4a6411d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c0ccb20 con 0x7f4a5c0cf280 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.238+0000 7f4a6411d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c0cbf10 con 0x7f4a5c15cc20 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.238+0000 7f4a6411d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c0b3110 con 0x7f4a5c0cd870 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a616b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a5c0cd870 0x7f4a5c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a626ba700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4a5c0cf280 0x7f4a5c0c5590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48554/0 (socket says 172.21.15.16:48554) 2023-10-06T11:42:28.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a626ba700 1 -- 172.21.15.16:0/2281028483 learned_addr learned my addr 172.21.15.16:0/2281028483 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a616b8700 1 -- 172.21.15.16:0/2281028483 >> v1:172.21.15.177:6789/0 conn(0x7f4a5c15cc20 legacy=0x7f4a5c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a616b8700 1 -- 172.21.15.16:0/2281028483 >> v1:172.21.15.16:6789/0 conn(0x7f4a5c0cf280 legacy=0x7f4a5c0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a60eb7700 1 -- 172.21.15.16:0/2281028483 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609308565 0 0) 0x7f4a5c0ccb20 con 0x7f4a5c0cf280 2023-10-06T11:42:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a616b8700 1 -- 172.21.15.16:0/2281028483 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c0cc1f0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a616b8700 1 --2- 172.21.15.16:0/2281028483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c0ced40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098e0 tx=0x7f4a500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb612a038299509e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a60eb7700 1 -- 172.21.15.16:0/2281028483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50009e80 con 0x7f4a5c0cd870 2023-10-06T11:42:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a60eb7700 1 -- 172.21.15.16:0/2281028483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5000b860 con 0x7f4a5c0cd870 2023-10-06T11:42:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.239+0000 7f4a60eb7700 1 -- 172.21.15.16:0/2281028483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000bbe0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 -- 172.21.15.16:0/2281028483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 msgr2=0x7f4a5c0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 --2- 172.21.15.16:0/2281028483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c0ced40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a500098e0 tx=0x7f4a500091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 -- 172.21.15.16:0/2281028483 shutdown_connections 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 --2- 172.21.15.16:0/2281028483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c0ced40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 -- 172.21.15.16:0/2281028483 >> 172.21.15.16:0/2281028483 conn(0x7f4a5c099b40 msgr2=0x7f4a5c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 -- 172.21.15.16:0/2281028483 shutdown_connections 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.240+0000 7f4a6411d700 1 -- 172.21.15.16:0/2281028483 wait complete. 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.241+0000 7f4a6411d700 1 Processor -- start 2023-10-06T11:42:28.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.241+0000 7f4a6411d700 1 -- start start 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0cf280 0x7f4a5c09d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c15cc20 0x7f4a5c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c09e240 con 0x7f4a5c0cd870 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c169d80 con 0x7f4a5c15cc20 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a6411d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c09df60 con 0x7f4a5c0cf280 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58120/0 (socket says 172.21.15.16:58120) 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 -- 172.21.15.16:0/1216435478 learned_addr learned my addr 172.21.15.16:0/1216435478 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a626ba700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c15cc20 0x7f4a5c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a616b8700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0cf280 0x7f4a5c09d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 -- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0cf280 msgr2=0x7f4a5c09d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0cf280 0x7f4a5c09d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 -- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c15cc20 msgr2=0x7f4a5c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c15cc20 0x7f4a5c158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 -- 172.21.15.16:0/1216435478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a50009420 con 0x7f4a5c0cd870 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.242+0000 7f4a61eb9700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400c590 tx=0x7f4a5400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.243+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011590 con 0x7f4a5c0cd870 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.243+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a54011710 con 0x7f4a5c0cd870 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.243+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c0c8c50 con 0x7f4a5c0cd870 2023-10-06T11:42:28.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.243+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011ad0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.243+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c0c9210 con 0x7f4a5c0cd870 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.244+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c0c9430 con 0x7f4a5c0cd870 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.244+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a5401b3b0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.244+0000 7f4a59ffb700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.244+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4a54083ac0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.245+0000 7f4a616b8700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.245+0000 7f4a616b8700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a500093f0 tx=0x7f4a50017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.249+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a54050bd0 con 0x7f4a5c0cd870 2023-10-06T11:42:28.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.522+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7f4a5c04c380 con 0x7f4a5c0cd870 2023-10-06T11:42:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.540+0000 7f4a59ffb700 1 -- 172.21.15.16:0/1216435478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4a54017070 con 0x7f4a5c0cd870 2023-10-06T11:42:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 msgr2=0x7f4a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4a500093f0 tx=0x7f4a50017000 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 msgr2=0x7f4a5c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400c590 tx=0x7f4a5400c8a0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 shutdown_connections 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a5c0cd870 0x7f4a5c15d2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4a5c15cc20 0x7f4a5c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 --2- 172.21.15.16:0/1216435478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4a5c0cf280 0x7f4a5c09d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 >> 172.21.15.16:0/1216435478 conn(0x7f4a5c099b40 msgr2=0x7f4a5c0ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 shutdown_connections 2023-10-06T11:42:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.542+0000 7f4a6411d700 1 -- 172.21.15.16:0/1216435478 wait complete. 2023-10-06T11:42:28.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:42:28.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.676+0000 7fa4949aa700 1 Processor -- start 2023-10-06T11:42:28.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.676+0000 7fa4949aa700 1 -- start start 2023-10-06T11:42:28.678 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa4949aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4901594a0 0x7fa4900ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa4949aa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa4900ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa4949aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4900cd900 con 0x7fa4900cd4a0 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa4949aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4900cffc0 con 0x7fa4901594a0 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa4949aa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900cf3b0 con 0x7fa4900ced70 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa48f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4901594a0 0x7fa4900ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa48ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa4900cd4a0 0x7fa4900c54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38046/0 (socket says 172.21.15.16:38046) 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa48ffff700 1 -- 172.21.15.16:0/4059879130 learned_addr learned my addr 172.21.15.16:0/4059879130 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.677+0000 7fa48effd700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa4900ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48f7fe700 1 -- 172.21.15.16:0/4059879130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 msgr2=0x7fa4900ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:28.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa4900ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48f7fe700 1 -- 172.21.15.16:0/4059879130 >> v1:172.21.15.16:6789/0 conn(0x7fa4900cd4a0 legacy=0x7fa4900c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48e7fc700 1 -- 172.21.15.16:0/4059879130 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684365416 0 0) 0x7fa488001490 con 0x7fa4900cd4a0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48f7fe700 1 -- 172.21.15.16:0/4059879130 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4900cf690 con 0x7fa4901594a0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48effd700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa4900ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa4900ce830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa4800098d0 tx=0x7fa480009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc4408dbbc4cc4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48e7fc700 1 -- 172.21.15.16:0/4059879130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480009e70 con 0x7fa4901594a0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48e7fc700 1 -- 172.21.15.16:0/4059879130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa480005c10 con 0x7fa4901594a0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa48e7fc700 1 -- 172.21.15.16:0/4059879130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000b650 con 0x7fa4901594a0 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.678+0000 7fa4949aa700 1 -- 172.21.15.16:0/4059879130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 msgr2=0x7fa4900ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa4900ce830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa4800098d0 tx=0x7fa480009120 comp rx=0 tx=0).stop 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 -- 172.21.15.16:0/4059879130 shutdown_connections 2023-10-06T11:42:28.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa4900ce830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 --2- 172.21.15.16:0/4059879130 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa4900ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 -- 172.21.15.16:0/4059879130 >> 172.21.15.16:0/4059879130 conn(0x7fa490089f90 msgr2=0x7fa4900998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 -- 172.21.15.16:0/4059879130 shutdown_connections 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.679+0000 7fa4949aa700 1 -- 172.21.15.16:0/4059879130 wait complete. 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.680+0000 7fa4949aa700 1 Processor -- start 2023-10-06T11:42:28.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.680+0000 7fa4949aa700 1 -- start start 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 0x7fa49016ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa49015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49016b270 con 0x7fa4901594a0 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49015c670 con 0x7fa4900cd4a0 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4949aa700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49016b5b0 con 0x7fa4900ced70 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37022/0 (socket says 172.21.15.16:37022) 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 -- 172.21.15.16:0/3410312205 learned_addr learned my addr 172.21.15.16:0/3410312205 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:28.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48effd700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa49015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 0x7fa49016ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 -- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 msgr2=0x7fa49015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa49015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 -- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 msgr2=0x7fa49016ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 0x7fa49016ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 -- 172.21.15.16:0/3410312205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800093c0 con 0x7fa4901594a0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 0x7fa49016ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa48ffff700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cc40 tx=0x7fa488001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.681+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800a870 con 0x7fa4901594a0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.682+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48800a9f0 con 0x7fa4901594a0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.682+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49015bdd0 con 0x7fa4901594a0 2023-10-06T11:42:28.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.682+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800adb0 con 0x7fa4901594a0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.682+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49015c390 con 0x7fa4901594a0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.683+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa490089530 con 0x7fa4901594a0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.683+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa488007300 con 0x7fa4901594a0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.683+0000 7fa4777fe700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.683+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4880847c0 con 0x7fa4901594a0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.684+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.684+0000 7fa48f7fe700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa480009160 tx=0x7fa480016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.688+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa488051950 con 0x7fa4901594a0 2023-10-06T11:42:28.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.883+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa4900cd900 con 0x7fa4901594a0 2023-10-06T11:42:28.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.886+0000 7fa4777fe700 1 -- 172.21.15.16:0/3410312205 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa4900cd900 con 0x7fa4901594a0 2023-10-06T11:42:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 msgr2=0x7fa47c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 0x7fa47c0693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa480009160 tx=0x7fa480016000 comp rx=0 tx=0).stop 2023-10-06T11:42:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 msgr2=0x7fa49015e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cc40 tx=0x7fa488001970 comp rx=0 tx=0).stop 2023-10-06T11:42:28.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 shutdown_connections 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4901594a0 0x7fa49015e7e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fa47c066f30 0x7fa47c0693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4900cd4a0 0x7fa49016ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 --2- 172.21.15.16:0/3410312205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4900ced70 0x7fa49015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 >> 172.21.15.16:0/3410312205 conn(0x7fa490089f90 msgr2=0x7fa4900cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 shutdown_connections 2023-10-06T11:42:28.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:28.888+0000 7fa4949aa700 1 -- 172.21.15.16:0/3410312205 wait complete. 2023-10-06T11:42:28.901 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:42:28.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:42:28.940 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:42:28.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:42:28.986 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-10-06T11:42:28.986 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:42:28.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:29.022 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:42:29.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:42:29.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:29.023 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:42:29.078 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi177.front.sepia.ceph.com... 2023-10-06T11:42:29.078 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:42:29.079 DEBUG:teuthology.orchestra.run.smithi177:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:29.116 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:42:29.117 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:42:29.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:29.117 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T11:42:29.174 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-10-06T11:42:29.174 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:42:29.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T11:42:29.216 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:42:29.217 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:42:29.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:29.217 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T11:42:29.272 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi177.front.sepia.ceph.com... 2023-10-06T11:42:29.272 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:42:29.272 DEBUG:teuthology.orchestra.run.smithi177:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T11:42:29.315 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T11:42:29.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:42:29.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:29.316 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T11:42:29.372 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T11:42:29.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:29.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.490+0000 7fce61846700 1 Processor -- start 2023-10-06T11:42:29.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.490+0000 7fce61846700 1 -- start start 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fce5c0cde50 0x7fce5c0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce5c0af2f0 0x7fce5c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c0ce220 con 0x7fce5c0af2f0 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c06c060 con 0x7fce5c0cde50 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61846700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0af730 con 0x7fce5c0b1c30 2023-10-06T11:42:29.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61045700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce5c0af2f0 0x7fce5c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61045700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce5c0af2f0 0x7fce5c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37032/0 (socket says 172.21.15.16:37032) 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce61045700 1 -- 172.21.15.16:0/160114603 learned_addr learned my addr 172.21.15.16:0/160114603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce5bfff700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.491+0000 7fce60844700 1 --2- 172.21.15.16:0/160114603 >> v2:172.21.15.177:3300/0 conn(0x7fce5c0cde50 0x7fce5c0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 -- 172.21.15.16:0/160114603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 msgr2=0x7fce5c0aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c0aedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 -- 172.21.15.16:0/160114603 >> v2:172.21.15.177:3300/0 conn(0x7fce5c0cde50 msgr2=0x7fce5c0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 --2- 172.21.15.16:0/160114603 >> v2:172.21.15.177:3300/0 conn(0x7fce5c0cde50 0x7fce5c0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 -- 172.21.15.16:0/160114603 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c06b4e0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61045700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c0b04b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce5c06e980 tx=0x7fce4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea83ea194af038ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce5b7fe700 1 -- 172.21.15.16:0/160114603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00b640 con 0x7fce5c0af2f0 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce5b7fe700 1 -- 172.21.15.16:0/160114603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4c00b7a0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce5b7fe700 1 -- 172.21.15.16:0/160114603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00bb60 con 0x7fce5c0af2f0 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 -- 172.21.15.16:0/160114603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 msgr2=0x7fce5c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c0b04b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce5c06e980 tx=0x7fce4c009250 comp rx=0 tx=0).stop 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 -- 172.21.15.16:0/160114603 shutdown_connections 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c0b04b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 --2- 172.21.15.16:0/160114603 >> v2:172.21.15.177:3300/0 conn(0x7fce5c0cde50 0x7fce5c0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 --2- 172.21.15.16:0/160114603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c0aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 -- 172.21.15.16:0/160114603 >> 172.21.15.16:0/160114603 conn(0x7fce5c0675c0 msgr2=0x7fce5c0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.492+0000 7fce61846700 1 -- 172.21.15.16:0/160114603 shutdown_connections 2023-10-06T11:42:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.493+0000 7fce61846700 1 -- 172.21.15.16:0/160114603 wait complete. 2023-10-06T11:42:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 Processor -- start 2023-10-06T11:42:29.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 -- start start 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c16ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce5c0cde50 0x7fce5c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c16e7f0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.494+0000 7fce61846700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c153e40 con 0x7fce5c0cde50 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce61846700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c14e840 con 0x7fce5c0b1c30 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37044/0 (socket says 172.21.15.16:37044) 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 -- 172.21.15.16:0/3600759268 learned_addr learned my addr 172.21.15.16:0/3600759268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:29.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce61045700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce5c0cde50 0x7fce5c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce5bfff700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c16ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 -- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 msgr2=0x7fce5c16ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c16ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 -- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce5c0cde50 msgr2=0x7fce5c14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce5c0cde50 0x7fce5c14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 -- 172.21.15.16:0/3600759268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0094f0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce60844700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce54009e50 tx=0x7fce5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400caa0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5400f040 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5400ce30 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c14eb20 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.495+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c14d400 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.496+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c14ed10 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.497+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce54014690 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.497+0000 7fce58ff9700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 0x7fce480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.497+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fce54084ff0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.497+0000 7fce5bfff700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 0x7fce480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.498+0000 7fce5bfff700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 0x7fce480693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce5c171690 tx=0x7fce500091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.501+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce54052100 con 0x7fce5c0af2f0 2023-10-06T11:42:29.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.739+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce5c16ead0 con 0x7fce5c0af2f0 2023-10-06T11:42:29.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.740+0000 7fce58ff9700 1 -- 172.21.15.16:0/3600759268 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fce54055720 con 0x7fce5c0af2f0 2023-10-06T11:42:29.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:29.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 msgr2=0x7fce480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 0x7fce480693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fce5c171690 tx=0x7fce500091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 msgr2=0x7fce5c146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce54009e50 tx=0x7fce5400b040 comp rx=0 tx=0).stop 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 shutdown_connections 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce5c0af2f0 0x7fce5c146110 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fce48066f30 0x7fce480693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fce5c0cde50 0x7fce5c14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 --2- 172.21.15.16:0/3600759268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fce5c0b1c30 0x7fce5c16ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 >> 172.21.15.16:0/3600759268 conn(0x7fce5c0675c0 msgr2=0x7fce5c155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 shutdown_connections 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.742+0000 7fce61846700 1 -- 172.21.15.16:0/3600759268 wait complete. 2023-10-06T11:42:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:29.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:29.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.873+0000 7f5ea67ee700 1 Processor -- start 2023-10-06T11:42:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.873+0000 7f5ea67ee700 1 -- start start 2023-10-06T11:42:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5ea67ee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5ea01595b0 0x7f5ea00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5ea67ee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00cf780 con 0x7f5ea0159980 2023-10-06T11:42:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5ea67ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00cbf20 con 0x7f5ea00cf380 2023-10-06T11:42:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5ea67ee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00cc200 con 0x7f5ea01595b0 2023-10-06T11:42:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5e9f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ea0159980 0x7f5ea00cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38060/0 (socket says 172.21.15.16:38060) 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5e9f7fe700 1 -- 172.21.15.16:0/1834977727 learned_addr learned my addr 172.21.15.16:0/1834977727 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.874+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/1834977727 >> v2:172.21.15.177:3300/0 conn(0x7f5ea01595b0 0x7f5ea00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254629561 0 0) 0x7f5ea00cf780 con 0x7f5ea0159980 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e8c003570 con 0x7f5ea0159980 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 566881044 0 0) 0x7f5e8c003570 con 0x7f5ea0159980 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 >> v2:172.21.15.177:3300/0 conn(0x7f5ea01595b0 msgr2=0x7f5ea00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 --2- 172.21.15.16:0/1834977727 >> v2:172.21.15.177:3300/0 conn(0x7f5ea01595b0 0x7f5ea00cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 >> v1:172.21.15.177:6789/0 conn(0x7f5ea00cf380 legacy=0x7f5ea00c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00cc4e0 con 0x7f5ea0159980 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f5e90001470 con 0x7f5ea0159980 2023-10-06T11:42:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 >> v1:172.21.15.16:6789/0 conn(0x7f5ea0159980 legacy=0x7f5ea00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c003a60 0x7f5e8c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8c004810 0x7f5e8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 0x7f5e8c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00cc200 con 0x7f5e8c0051c0 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c00c160 con 0x7f5e8c003a60 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c00c320 con 0x7f5e8c004810 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 0x7f5e8c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c003a60 0x7f5e8c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9f7fe700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8c004810 0x7f5e8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 -- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8c004810 msgr2=0x7f5e8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5e8c004810 0x7f5e8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 -- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c003a60 msgr2=0x7f5e8c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c003a60 0x7f5e8c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5ea4d8b700 1 -- 172.21.15.16:0/1834977727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c003570 con 0x7f5e8c0051c0 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.875+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c003a60 0x7f5e8c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea4d8b700 1 -- 172.21.15.16:0/1834977727 reap_dead start 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea4d8b700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 0x7f5e8c00ba60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e98001fc0 tx=0x7f5e98001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de57861b0ef630f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98001950 con 0x7f5e8c0051c0 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800c8b0 con 0x7f5e8c0051c0 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5e9effd700 1 -- 172.21.15.16:0/1834977727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800cc90 con 0x7f5e8c0051c0 2023-10-06T11:42:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 -- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 msgr2=0x7f5e8c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 0x7f5e8c00ba60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5e98001fc0 tx=0x7f5e98001640 comp rx=0 tx=0).stop 2023-10-06T11:42:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 -- 172.21.15.16:0/1834977727 shutdown_connections 2023-10-06T11:42:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/1834977727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e8c0051c0 0x7f5e8c00ba60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 -- 172.21.15.16:0/1834977727 >> 172.21.15.16:0/1834977727 conn(0x7f5ea0099b40 msgr2=0x7f5ea0087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 -- 172.21.15.16:0/1834977727 shutdown_connections 2023-10-06T11:42:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.876+0000 7f5ea67ee700 1 -- 172.21.15.16:0/1834977727 wait complete. 2023-10-06T11:42:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.877+0000 7f5ea67ee700 1 Processor -- start 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 -- start start 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c0051c0 0x7f5ea00c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ea014d580 0x7f5ea0152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00ca2d0 con 0x7f5ea00c8160 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00c7a90 con 0x7f5e8c0051c0 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5ea67ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0155b00 con 0x7f5ea014d580 2023-10-06T11:42:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37064/0 (socket says 172.21.15.16:37064) 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.878+0000 7f5e9f7fe700 1 -- 172.21.15.16:0/781101555 learned_addr learned my addr 172.21.15.16:0/781101555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c0051c0 0x7f5ea00c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5ea4d8b700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ea014d580 0x7f5ea0152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 -- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ea014d580 msgr2=0x7f5ea0152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ea014d580 0x7f5ea0152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 -- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c0051c0 msgr2=0x7f5ea00c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c0051c0 0x7f5ea00c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 -- 172.21.15.16:0/781101555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00ca520 con 0x7f5ea00c8160 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e9f7fe700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007390 tx=0x7f5e90001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90018030 con 0x7f5ea00c8160 2023-10-06T11:42:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90007800 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00ca800 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9000fb00 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.879+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea01576c0 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.880+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea004c380 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.880+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e9001b070 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.881+0000 7f5e97fff700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.881+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5e90019070 con 0x7f5ea00c8160 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.881+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.882+0000 7f5e9ffff700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 0x7f5e800693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009170 tx=0x7f5e88008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:29.885+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e900516b0 con 0x7f5ea00c8160 2023-10-06T11:42:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.078+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ea00cc7c0 con 0x7f5ea00c8160 2023-10-06T11:42:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.079+0000 7f5e97fff700 1 -- 172.21.15.16:0/781101555 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f5e90054cd0 con 0x7f5ea00c8160 2023-10-06T11:42:30.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:30.080 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 msgr2=0x7f5e800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 0x7f5e800693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009170 tx=0x7f5e88008040 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 msgr2=0x7f5ea0157180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5e90007390 tx=0x7f5e90001e50 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 shutdown_connections 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5e80066f30 0x7f5e800693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5ea014d580 0x7f5ea0152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5e8c0051c0 0x7f5ea00c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 --2- 172.21.15.16:0/781101555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ea00c8160 0x7f5ea0157180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 >> 172.21.15.16:0/781101555 conn(0x7f5ea0099b40 msgr2=0x7f5ea00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 shutdown_connections 2023-10-06T11:42:30.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.081+0000 7f5ea67ee700 1 -- 172.21.15.16:0/781101555 wait complete. 2023-10-06T11:42:30.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:41:53.661288+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:41:54.874734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:42:30.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.216+0000 7f53e4d97700 1 Processor -- start 2023-10-06T11:42:30.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.216+0000 7f53e4d97700 1 -- start start 2023-10-06T11:42:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53e4d97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53e0159740 0x7f53e00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53e4d97700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f53e00cda10 0x7f53e00b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53e4d97700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00cead0 con 0x7f53e0159b10 2023-10-06T11:42:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53e4d97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00affc0 con 0x7f53e0159740 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53e4d97700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00cdec0 con 0x7f53e00cda10 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53e0159740 0x7f53e00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53e0159740 0x7f53e00b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37066/0 (socket says 172.21.15.16:37066) 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 -- 172.21.15.16:0/1294853755 learned_addr learned my addr 172.21.15.16:0/1294853755 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53ddd9b700 1 --2- 172.21.15.16:0/1294853755 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cda10 0x7f53e00b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 -- 172.21.15.16:0/1294853755 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cda10 msgr2=0x7f53e00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 --2- 172.21.15.16:0/1294853755 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cda10 0x7f53e00b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 -- 172.21.15.16:0/1294853755 >> v1:172.21.15.177:6789/0 conn(0x7f53e0159b10 legacy=0x7f53e015d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 -- 172.21.15.16:0/1294853755 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00ce1a0 con 0x7f53e0159740 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53de59c700 1 --2- 172.21.15.16:0/1294853755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159740 0x7f53e00b3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53e006bd00 tx=0x7f53d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69c5b68ebfc8391 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.217+0000 7f53dd59a700 1 -- 172.21.15.16:0/1294853755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d0009e70 con 0x7f53e0159740 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53dd59a700 1 -- 172.21.15.16:0/1294853755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000b860 con 0x7f53e0159740 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53dd59a700 1 -- 172.21.15.16:0/1294853755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000bc00 con 0x7f53e0159740 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 -- 172.21.15.16:0/1294853755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159740 msgr2=0x7f53e00b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 --2- 172.21.15.16:0/1294853755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159740 0x7f53e00b3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53e006bd00 tx=0x7f53d0009250 comp rx=0 tx=0).stop 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 -- 172.21.15.16:0/1294853755 shutdown_connections 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 --2- 172.21.15.16:0/1294853755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159740 0x7f53e00b3990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 --2- 172.21.15.16:0/1294853755 >> v2:172.21.15.177:3300/0 conn(0x7f53e00cda10 0x7f53e00b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 -- 172.21.15.16:0/1294853755 >> 172.21.15.16:0/1294853755 conn(0x7f53e0099a40 msgr2=0x7f53e006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 -- 172.21.15.16:0/1294853755 shutdown_connections 2023-10-06T11:42:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.218+0000 7f53e4d97700 1 -- 172.21.15.16:0/1294853755 wait complete. 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.219+0000 7f53e4d97700 1 Processor -- start 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.219+0000 7f53e4d97700 1 -- start start 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cda10 0x7f53e015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e0159740 0x7f53e015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e015dce0 con 0x7f53e0159b10 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e015d440 con 0x7f53e00cda10 2023-10-06T11:42:30.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53e4d97700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e015d720 con 0x7f53e0159740 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37080/0 (socket says 172.21.15.16:37080) 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53ded9d700 1 -- 172.21.15.16:0/3023965762 learned_addr learned my addr 172.21.15.16:0/3023965762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53ddd9b700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e0159740 0x7f53e015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.220+0000 7f53de59c700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cda10 0x7f53e015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 -- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e0159740 msgr2=0x7f53e015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e0159740 0x7f53e015c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 -- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cda10 msgr2=0x7f53e015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cda10 0x7f53e015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 -- 172.21.15.16:0/3023965762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00094f0 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53ded9d700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c840 tx=0x7f53d8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012630 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d80127b0 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e015da00 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8012b70 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.221+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e009e1f0 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.222+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e009a3b0 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.222+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53d8003030 con 0x7f53e0159b10 2023-10-06T11:42:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.223+0000 7f53d67fc700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 0x7f53c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.223+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f53d80846b0 con 0x7f53e0159b10 2023-10-06T11:42:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.223+0000 7f53de59c700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 0x7f53c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.223+0000 7f53de59c700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 0x7f53c00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53e006bd00 tx=0x7f53d0017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.227+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53e009a3b0 con 0x7f53e0159b10 2023-10-06T11:42:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.417+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53e009a3b0 con 0x7f53e0159b10 2023-10-06T11:42:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.418+0000 7f53d67fc700 1 -- 172.21.15.16:0/3023965762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f53e009a3b0 con 0x7f53e0159b10 2023-10-06T11:42:30.420 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 msgr2=0x7f53c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 0x7f53c00693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53e006bd00 tx=0x7f53d0017000 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 msgr2=0x7f53e015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53d800c840 tx=0x7f53d8001670 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 shutdown_connections 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53e0159b10 0x7f53e015c5f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f53c0066f30 0x7f53c00693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f53e00cda10 0x7f53e015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 --2- 172.21.15.16:0/3023965762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f53e0159740 0x7f53e015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 >> 172.21.15.16:0/3023965762 conn(0x7f53e0099a40 msgr2=0x7f53e00b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 shutdown_connections 2023-10-06T11:42:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.420+0000 7f53e4d97700 1 -- 172.21.15.16:0/3023965762 wait complete. 2023-10-06T11:42:30.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.555+0000 7f2695735700 1 Processor -- start 2023-10-06T11:42:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.555+0000 7f2695735700 1 -- start start 2023-10-06T11:42:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2695735700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26900cef40 0x7f26900ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2695735700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2695735700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900cffc0 con 0x7f26901594a0 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2695735700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26900cf3b0 con 0x7f26900cef40 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2695735700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900cf690 con 0x7f26900c64c0 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26900cef40 0x7f26900ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26900cef40 0x7f26900ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37090/0 (socket says 172.21.15.16:37090) 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 -- 172.21.15.16:0/1512200511 learned_addr learned my addr 172.21.15.16:0/1512200511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f2694f34700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 -- 172.21.15.16:0/1512200511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 msgr2=0x7f26900cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 -- 172.21.15.16:0/1512200511 >> v1:172.21.15.177:6789/0 conn(0x7f26901594a0 legacy=0x7f26900ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.556+0000 7f268f7fe700 1 -- 172.21.15.16:0/1512200511 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26900cf970 con 0x7f26900cef40 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f26900ccf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26780099b0 tx=0x7f2678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd3856a486b7513 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f268effd700 1 -- 172.21.15.16:0/1512200511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800b860 con 0x7f26900cef40 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f268effd700 1 -- 172.21.15.16:0/1512200511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267800b9c0 con 0x7f26900cef40 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f268effd700 1 -- 172.21.15.16:0/1512200511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267800bd60 con 0x7f26900cef40 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 -- 172.21.15.16:0/1512200511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 msgr2=0x7f26900ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f26900ccf60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f26780099b0 tx=0x7f2678009250 comp rx=0 tx=0).stop 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 -- 172.21.15.16:0/1512200511 shutdown_connections 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 --2- 172.21.15.16:0/1512200511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f26900ccf60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 -- 172.21.15.16:0/1512200511 >> 172.21.15.16:0/1512200511 conn(0x7f2690089f90 msgr2=0x7f26900998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 -- 172.21.15.16:0/1512200511 shutdown_connections 2023-10-06T11:42:30.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.557+0000 7f2695735700 1 -- 172.21.15.16:0/1512200511 wait complete. 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.558+0000 7f2695735700 1 Processor -- start 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 -- start start 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26901594a0 0x7f2690153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269016a350 con 0x7f26900cef40 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269016a630 con 0x7f26901594a0 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f2695735700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269016a910 con 0x7f26900c64c0 2023-10-06T11:42:30.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f268f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37092/0 (socket says 172.21.15.16:37092) 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.559+0000 7f268f7fe700 1 -- 172.21.15.16:0/1726613273 learned_addr learned my addr 172.21.15.16:0/1726613273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 -- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 msgr2=0x7f26900b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 -- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26901594a0 msgr2=0x7f2690153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26901594a0 0x7f2690153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 -- 172.21.15.16:0/1726613273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780094f0 con 0x7f26900cef40 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f268f7fe700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2678009290 tx=0x7f2678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678024440 con 0x7f26900cef40 2023-10-06T11:42:30.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26780245c0 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678024980 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269016abf0 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.560+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26900b53a0 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.561+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26900b5590 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.562+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f267801a420 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.562+0000 7f2687fff700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.562+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2678086510 con 0x7f26900cef40 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.562+0000 7f268ffff700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.563+0000 7f268ffff700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 0x7f26700693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2680001640 tx=0x7f2680007b90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.566+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2678053620 con 0x7f26900cef40 2023-10-06T11:42:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.760+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26900cfc50 con 0x7f26900cef40 2023-10-06T11:42:30.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.761+0000 7f2687fff700 1 -- 172.21.15.16:0/1726613273 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2678056c40 con 0x7f26900cef40 2023-10-06T11:42:30.762 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 msgr2=0x7f26700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 0x7f26700693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2680001640 tx=0x7f2680007b90 comp rx=0 tx=0).stop 2023-10-06T11:42:30.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 msgr2=0x7f269015b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2678009290 tx=0x7f2678017000 comp rx=0 tx=0).stop 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 shutdown_connections 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2670066f30 0x7f26700693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26901594a0 0x7f2690153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26900c64c0 0x7f26900b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 --2- 172.21.15.16:0/1726613273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26900cef40 0x7f269015b530 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 >> 172.21.15.16:0/1726613273 conn(0x7f2690089f90 msgr2=0x7f2690151a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 shutdown_connections 2023-10-06T11:42:30.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.763+0000 7f2695735700 1 -- 172.21.15.16:0/1726613273 wait complete. 2023-10-06T11:42:30.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:42:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.896+0000 7f9f3106b700 1 Processor -- start 2023-10-06T11:42:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.896+0000 7f9f3106b700 1 -- start start 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f3106b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f2c15bba0 0x7f9f2c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f3106b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9f2c0cd070 0x7f9f2c15a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f3106b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2c0ccbe0 con 0x7f9f2c0cb0b0 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f3106b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0ca160 con 0x7f9f2c15bba0 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f3106b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0cc2d0 con 0x7f9f2c0cd070 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f2a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f2c15bba0 0x7f9f2c0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f2ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f2c0cb0b0 0x7f9f2c0c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38072/0 (socket says 172.21.15.16:38072) 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f2ad9d700 1 -- 172.21.15.16:0/1383950195 learned_addr learned my addr 172.21.15.16:0/1383950195 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.897+0000 7f9f2b59e700 1 --2- 172.21.15.16:0/1383950195 >> v2:172.21.15.177:3300/0 conn(0x7f9f2c0cd070 0x7f9f2c15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f2a59c700 1 -- 172.21.15.16:0/1383950195 >> v2:172.21.15.177:3300/0 conn(0x7f9f2c0cd070 msgr2=0x7f9f2c15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f2a59c700 1 --2- 172.21.15.16:0/1383950195 >> v2:172.21.15.177:3300/0 conn(0x7f9f2c0cd070 0x7f9f2c15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f2a59c700 1 -- 172.21.15.16:0/1383950195 >> v1:172.21.15.16:6789/0 conn(0x7f9f2c0cb0b0 legacy=0x7f9f2c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f2a59c700 1 -- 172.21.15.16:0/1383950195 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c0c9310 con 0x7f9f2c15bba0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f29d9b700 1 -- 172.21.15.16:0/1383950195 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138235835 0 0) 0x7f9f2c0ccbe0 con 0x7f9f2c0cb0b0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f2a59c700 1 --2- 172.21.15.16:0/1383950195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c0cb480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0b44b0 tx=0x7f9f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b28b91a507f85ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f29d9b700 1 -- 172.21.15.16:0/1383950195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011630 con 0x7f9f2c15bba0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f29d9b700 1 -- 172.21.15.16:0/1383950195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14011790 con 0x7f9f2c15bba0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f29d9b700 1 -- 172.21.15.16:0/1383950195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011b30 con 0x7f9f2c15bba0 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 -- 172.21.15.16:0/1383950195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 msgr2=0x7f9f2c0cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 --2- 172.21.15.16:0/1383950195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c0cb480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0b44b0 tx=0x7f9f140091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 -- 172.21.15.16:0/1383950195 shutdown_connections 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 --2- 172.21.15.16:0/1383950195 >> v2:172.21.15.177:3300/0 conn(0x7f9f2c0cd070 0x7f9f2c15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 --2- 172.21.15.16:0/1383950195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c0cb480 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.898+0000 7f9f3106b700 1 -- 172.21.15.16:0/1383950195 >> 172.21.15.16:0/1383950195 conn(0x7f9f2c067c90 msgr2=0x7f9f2c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.899+0000 7f9f3106b700 1 -- 172.21.15.16:0/1383950195 shutdown_connections 2023-10-06T11:42:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.899+0000 7f9f3106b700 1 -- 172.21.15.16:0/1383950195 wait complete. 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.900+0000 7f9f3106b700 1 Processor -- start 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.900+0000 7f9f3106b700 1 -- start start 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 0x7f9f2c0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 0x7f9f2c148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06d520 con 0x7f9f2c15bba0 2023-10-06T11:42:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06e0a0 con 0x7f9f2c0cd070 2023-10-06T11:42:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06d770 con 0x7f9f2c0cb0b0 2023-10-06T11:42:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37100/0 (socket says 172.21.15.16:37100) 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 -- 172.21.15.16:0/3710933556 learned_addr learned my addr 172.21.15.16:0/3710933556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2a59c700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 0x7f9f2c148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2ad9d700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 0x7f9f2c0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 -- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 msgr2=0x7f9f2c0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 0x7f9f2c0c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 -- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 msgr2=0x7f9f2c148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 0x7f9f2c148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 -- 172.21.15.16:0/3710933556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009420 con 0x7f9f2c15bba0 2023-10-06T11:42:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2ad9d700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 0x7f9f2c0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2a59c700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 0x7f9f2c148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f2b59e700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009e90 tx=0x7f9f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00cc20 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c00f040 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c06da50 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c014480 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.901+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c0c8260 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.902+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2c06e210 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.903+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f1c014830 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.903+0000 7f9f22ffd700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.903+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9f1c085150 con 0x7f9f2c15bba0 2023-10-06T11:42:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.903+0000 7f9f2ad9d700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.904+0000 7f9f2ad9d700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007fd0 tx=0x7f9f24009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:30.907+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f1c055c50 con 0x7f9f2c15bba0 2023-10-06T11:42:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.100+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f2c04c380 con 0x7f9f2c15bba0 2023-10-06T11:42:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.101+0000 7f9f22ffd700 1 -- 172.21.15.16:0/3710933556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f1c0557e0 con 0x7f9f2c15bba0 2023-10-06T11:42:31.102 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:42:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 msgr2=0x7f9f0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9f24007fd0 tx=0x7f9f24009040 comp rx=0 tx=0).stop 2023-10-06T11:42:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 msgr2=0x7f9f2c156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009e90 tx=0x7f9f1c00b040 comp rx=0 tx=0).stop 2023-10-06T11:42:31.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 shutdown_connections 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2c15bba0 0x7f9f2c156af0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9f2c0cb0b0 0x7f9f2c0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 --2- 172.21.15.16:0/3710933556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f2c0cd070 0x7f9f2c148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 >> 172.21.15.16:0/3710933556 conn(0x7f9f2c067c90 msgr2=0x7f9f2c156040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 shutdown_connections 2023-10-06T11:42:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.103+0000 7f9f3106b700 1 -- 172.21.15.16:0/3710933556 wait complete. 2023-10-06T11:42:31.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:42:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.236+0000 7fb6dedc6700 1 Processor -- start 2023-10-06T11:42:31.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.236+0000 7fb6dedc6700 1 -- start start 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d80cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6d80cb660 0x7fb6d80cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 0x7fb6d815d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d815e270 con 0x7fb6d80cb660 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d813c9f0 con 0x7fb6d815d500 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dedc6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d806dff0 con 0x7fb6d80cb290 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6d80cb660 0x7fb6d80cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6d80cb660 0x7fb6d80cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37116/0 (socket says 172.21.15.16:37116) 2023-10-06T11:42:31.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 -- 172.21.15.16:0/304830489 learned_addr learned my addr 172.21.15.16:0/304830489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dd363700 1 --2- 172.21.15.16:0/304830489 >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 0x7fb6d815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dcb62700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d80cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 -- 172.21.15.16:0/304830489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 msgr2=0x7fb6d80cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d80cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 -- 172.21.15.16:0/304830489 >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 msgr2=0x7fb6d815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/304830489 >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 0x7fb6d815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 -- 172.21.15.16:0/304830489 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d815ae30 con 0x7fb6d80cb660 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dd363700 1 --2- 172.21.15.16:0/304830489 >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 0x7fb6d815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6dcb62700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.237+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d80cc600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6d81542c0 tx=0x7fb6c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96311ffc8d93da17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6d77fe700 1 -- 172.21.15.16:0/304830489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400b860 con 0x7fb6d80cb660 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6d77fe700 1 -- 172.21.15.16:0/304830489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c400b9c0 con 0x7fb6d80cb660 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6d77fe700 1 -- 172.21.15.16:0/304830489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400bd60 con 0x7fb6d80cb660 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 -- 172.21.15.16:0/304830489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 msgr2=0x7fb6d80cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d80cc600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6d81542c0 tx=0x7fb6c4009250 comp rx=0 tx=0).stop 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 -- 172.21.15.16:0/304830489 shutdown_connections 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/304830489 >> v2:172.21.15.177:3300/0 conn(0x7fb6d815d500 0x7fb6d815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d80cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/304830489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d80cc600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 -- 172.21.15.16:0/304830489 >> 172.21.15.16:0/304830489 conn(0x7fb6d8067c90 msgr2=0x7fb6d8068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 -- 172.21.15.16:0/304830489 shutdown_connections 2023-10-06T11:42:31.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.238+0000 7fb6dedc6700 1 -- 172.21.15.16:0/304830489 wait complete. 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.239+0000 7fb6dedc6700 1 Processor -- start 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.239+0000 7fb6dedc6700 1 -- start start 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d816e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6d815d500 0x7fb6d813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d813e4f0 con 0x7fb6d80cb660 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c6c10 con 0x7fb6d815d500 2023-10-06T11:42:31.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dedc6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d813e210 con 0x7fb6d80cb290 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6d7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37122/0 (socket says 172.21.15.16:37122) 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6d7fff700 1 -- 172.21.15.16:0/118684190 learned_addr learned my addr 172.21.15.16:0/118684190 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dcb62700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d816e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.240+0000 7fb6dd363700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6d815d500 0x7fb6d813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 -- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 msgr2=0x7fb6d816e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d816e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 -- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6d815d500 msgr2=0x7fb6d813f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6d815d500 0x7fb6d813f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 -- 172.21.15.16:0/118684190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40094f0 con 0x7fb6d80cb660 2023-10-06T11:42:31.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d7fff700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009290 tx=0x7fb6c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4026480 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c401f070 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d80c6e20 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4026830 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.241+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d80c73e0 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.242+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d80c75d0 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.242+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6c4026ba0 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.243+0000 7fb6d4ff9700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.243+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb6c4021070 con 0x7fb6d80cb660 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.243+0000 7fb6dcb62700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.243+0000 7fb6dcb62700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006f00 tx=0x7fb6cc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.247+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6c4053690 con 0x7fb6d80cb660 2023-10-06T11:42:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.438+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb6d804c380 con 0x7fb6d80cb660 2023-10-06T11:42:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.438+0000 7fb6d4ff9700 1 -- 172.21.15.16:0/118684190 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb6c4056cb0 con 0x7fb6d80cb660 2023-10-06T11:42:31.440 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 msgr2=0x7fb6bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 0x7fb6bc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc006f00 tx=0x7fb6cc008040 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 msgr2=0x7fb6d813db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009290 tx=0x7fb6c4017000 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 shutdown_connections 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb6bc066f30 0x7fb6bc0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6d815d500 0x7fb6d813f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6d80cb290 0x7fb6d816e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 --2- 172.21.15.16:0/118684190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6d80cb660 0x7fb6d813db30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 >> 172.21.15.16:0/118684190 conn(0x7fb6d8067c90 msgr2=0x7fb6d80ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 shutdown_connections 2023-10-06T11:42:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.440+0000 7fb6dedc6700 1 -- 172.21.15.16:0/118684190 wait complete. 2023-10-06T11:42:31.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.572+0000 7f0fd18f8700 1 Processor -- start 2023-10-06T11:42:31.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.572+0000 7f0fd18f8700 1 -- start start 2023-10-06T11:42:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fd18f8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fd18f8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0fcc06a760 0x7f0fcc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fd18f8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0ac430 con 0x7f0fcc156120 2023-10-06T11:42:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fd18f8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0abb00 con 0x7f0fcc06a760 2023-10-06T11:42:31.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fd18f8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0abde0 con 0x7f0fcc0af130 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fcb7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0fcc06a760 0x7f0fcc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fca7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fcb7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0fcc06a760 0x7f0fcc0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:46990/0 (socket says 172.21.15.16:46990) 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fca7fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:60680/0 (socket says 172.21.15.16:60680) 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fcb7fe700 1 -- 172.21.15.16:0/3312898339 learned_addr learned my addr 172.21.15.16:0/3312898339 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.573+0000 7f0fcaffd700 1 --1- 172.21.15.16:0/3312898339 >> v1:172.21.15.177:6789/0 conn(0x7f0fcc156120 0x7f0fcc06b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:59934/0 (socket says 172.21.15.16:59934) 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fcb7fe700 1 -- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 msgr2=0x7f0fcc1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fcb7fe700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fcb7fe700 1 -- 172.21.15.16:0/3312898339 >> v1:172.21.15.177:6789/0 conn(0x7f0fcc156120 legacy=0x7f0fcc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fcb7fe700 1 -- 172.21.15.16:0/3312898339 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0ac0c0 con 0x7f0fcc06a760 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fca7fc700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fcb7fe700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc06a760 0x7f0fcc0cc160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009f50 tx=0x7f0fc4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4b7e2c4d64f8c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fc9ffb700 1 -- 172.21.15.16:0/3312898339 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400e070 con 0x7f0fcc06a760 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fc9ffb700 1 -- 172.21.15.16:0/3312898339 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc400b9c0 con 0x7f0fcc06a760 2023-10-06T11:42:31.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.574+0000 7f0fc9ffb700 1 -- 172.21.15.16:0/3312898339 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40092e0 con 0x7f0fcc06a760 2023-10-06T11:42:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 -- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc06a760 msgr2=0x7f0fcc0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc06a760 0x7f0fcc0cc160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009f50 tx=0x7f0fc4009840 comp rx=0 tx=0).stop 2023-10-06T11:42:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 -- 172.21.15.16:0/3312898339 shutdown_connections 2023-10-06T11:42:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc06a760 0x7f0fcc0cc160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/3312898339 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc0af130 0x7f0fcc1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 -- 172.21.15.16:0/3312898339 >> 172.21.15.16:0/3312898339 conn(0x7f0fcc0675c0 msgr2=0x7f0fcc0679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 -- 172.21.15.16:0/3312898339 shutdown_connections 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.575+0000 7f0fd18f8700 1 -- 172.21.15.16:0/3312898339 wait complete. 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.576+0000 7f0fd18f8700 1 Processor -- start 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.576+0000 7f0fd18f8700 1 -- start start 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc0af130 0x7f0fcc0b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 0x7f0fcc1587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0afd10 con 0x7f0fcc06a760 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0afff0 con 0x7f0fcc0af130 2023-10-06T11:42:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fd18f8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0b02d0 con 0x7f0fcc156120 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37130/0 (socket says 172.21.15.16:37130) 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 -- 172.21.15.16:0/2224695260 learned_addr learned my addr 172.21.15.16:0/2224695260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fca7fc700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc0af130 0x7f0fcc0b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcb7fe700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 0x7f0fcc1587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 -- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 msgr2=0x7f0fcc1587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 0x7f0fcc1587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 -- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc0af130 msgr2=0x7f0fcc0b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc0af130 0x7f0fcc0b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcaffd700 1 -- 172.21.15.16:0/2224695260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4009420 con 0x7f0fcc06a760 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.577+0000 7f0fcb7fe700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 0x7f0fcc1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fcaffd700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00c560 tx=0x7f0fbc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc005a70 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc005bd0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc4009ae0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0070d0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.578+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc15c9f0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.579+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc0b06c0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.579+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fbc007440 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.579+0000 7f0fc2ffd700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 0x7f0fac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.580+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0fbc083ae0 con 0x7f0fcc06a760 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.580+0000 7f0fca7fc700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 0x7f0fac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.580+0000 7f0fca7fc700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 0x7f0fac0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc15c2a0 tx=0x7f0fb40091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.584+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fbc050bf0 con 0x7f0fcc06a760 2023-10-06T11:42:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.823+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fcc0b0490 con 0x7f0fcc06a760 2023-10-06T11:42:31.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.823+0000 7f0fc2ffd700 1 -- 172.21.15.16:0/2224695260 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f0fbc017020 con 0x7f0fcc06a760 2023-10-06T11:42:31.824 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:31.824 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 msgr2=0x7f0fac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 0x7f0fac0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc15c2a0 tx=0x7f0fb40091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 msgr2=0x7f0fcc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00c560 tx=0x7f0fbc007f70 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 shutdown_connections 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fcc06a760 0x7f0fcc15c4b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0fac066f30 0x7f0fac0693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0fcc156120 0x7f0fcc1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 --2- 172.21.15.16:0/2224695260 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fcc0af130 0x7f0fcc0b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 >> 172.21.15.16:0/2224695260 conn(0x7f0fcc0675c0 msgr2=0x7f0fcc155170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 shutdown_connections 2023-10-06T11:42:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.825+0000 7f0fd18f8700 1 -- 172.21.15.16:0/2224695260 wait complete. 2023-10-06T11:42:31.828 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:31.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:31.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.955+0000 7f2ac0d51700 1 Processor -- start 2023-10-06T11:42:31.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.955+0000 7f2ac0d51700 1 -- start start 2023-10-06T11:42:31.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.956+0000 7f2ac0d51700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2abc153cd0 0x7f2abc1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.956+0000 7f2ac0d51700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc15e4f0 con 0x7f2abc0675c0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.956+0000 7f2ac0d51700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc15d600 con 0x7f2abc06c410 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.956+0000 7f2ac0d51700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc067a30 con 0x7f2abc153cd0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab9d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2abc0675c0 0x7f2abc0af740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38082/0 (socket says 172.21.15.16:38082) 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab9d9b700 1 -- 172.21.15.16:0/3530461077 learned_addr learned my addr 172.21.15.16:0/3530461077 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2aba59c700 1 --2- 172.21.15.16:0/3530461077 >> v2:172.21.15.177:3300/0 conn(0x7f2abc153cd0 0x7f2abc1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691691967 0 0) 0x7f2abc15e4f0 con 0x7f2abc0675c0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa8003570 con 0x7f2abc0675c0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145096334 0 0) 0x7f2aa8003570 con 0x7f2abc0675c0 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 >> v2:172.21.15.177:3300/0 conn(0x7f2abc153cd0 msgr2=0x7f2abc1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 --2- 172.21.15.16:0/3530461077 >> v2:172.21.15.177:3300/0 conn(0x7f2abc153cd0 0x7f2abc1540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 >> v1:172.21.15.177:6789/0 conn(0x7f2abc06c410 legacy=0x7f2abc0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc15d8e0 con 0x7f2abc0675c0 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f2aa4001470 con 0x7f2abc0675c0 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 >> v1:172.21.15.16:6789/0 conn(0x7f2abc0675c0 legacy=0x7f2abc0af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2aa8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa8004810 0x7f2aa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa80051c0 0x7f2aa800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc067a30 con 0x7f2aa8003a60 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa800c0e0 con 0x7f2aa8004810 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa800c2a0 con 0x7f2aa80051c0 2023-10-06T11:42:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2aba59c700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2aa8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2ab9d9b700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa8004810 0x7f2aa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.957+0000 7f2abad9d700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa80051c0 0x7f2aa800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 -- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa80051c0 msgr2=0x7f2aa800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa80051c0 0x7f2aa800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 -- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa8004810 msgr2=0x7f2aa8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2aa8004810 0x7f2aa8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 -- 172.21.15.16:0/3530461077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8003570 con 0x7f2aa8003a60 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2abad9d700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2aa80051c0 0x7f2aa800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2abad9d700 1 -- 172.21.15.16:0/3530461077 reap_dead start 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2aba59c700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2aa8003e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009c00 tx=0x7f2aac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837537667b3dc984 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00e710 con 0x7f2aa8003a60 2023-10-06T11:42:31.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00e870 con 0x7f2aa8003a60 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ab959a700 1 -- 172.21.15.16:0/3530461077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00ec30 con 0x7f2aa8003a60 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ac0d51700 1 -- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 msgr2=0x7f2aa8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2aa8003e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009c00 tx=0x7f2aac0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ac0d51700 1 -- 172.21.15.16:0/3530461077 shutdown_connections 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/3530461077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2aa8003e30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.958+0000 7f2ac0d51700 1 -- 172.21.15.16:0/3530461077 >> 172.21.15.16:0/3530461077 conn(0x7f2abc0cd480 msgr2=0x7f2abc0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.959+0000 7f2ac0d51700 1 -- 172.21.15.16:0/3530461077 shutdown_connections 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.959+0000 7f2ac0d51700 1 -- 172.21.15.16:0/3530461077 wait complete. 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 Processor -- start 2023-10-06T11:42:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 -- start start 2023-10-06T11:42:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 0x7f2abc0ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 0x7f2abc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0ac6c0 con 0x7f2aa8003a60 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0b2ae0 con 0x7f2abc0b17a0 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2ac0d51700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0a9d40 con 0x7f2abc0abd10 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2aba59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2aba59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37152/0 (socket says 172.21.15.16:37152) 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.960+0000 7f2aba59c700 1 -- 172.21.15.16:0/2265833576 learned_addr learned my addr 172.21.15.16:0/2265833576 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2abad9d700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 0x7f2abc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ab9d9b700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 0x7f2abc0ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 -- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 msgr2=0x7f2abc0ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 0x7f2abc0ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 -- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 msgr2=0x7f2abc0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 0x7f2abc0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 -- 172.21.15.16:0/2265833576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0b2520 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ab9d9b700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 0x7f2abc0ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2abad9d700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 0x7f2abc0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2aba59c700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c0b0 tx=0x7f2aac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac022070 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00e9e0 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc0b2800 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac01c450 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.961+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc14f9b0 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.962+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc0aa020 con 0x7f2aa8003a60 2023-10-06T11:42:31.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.963+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2aac01c880 con 0x7f2aa8003a60 2023-10-06T11:42:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.963+0000 7f2ab27fc700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 0x7f2a9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.963+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2aac085ca0 con 0x7f2aa8003a60 2023-10-06T11:42:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.963+0000 7f2ab9d9b700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 0x7f2a9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.964+0000 7f2ab9d9b700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 0x7f2a9c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4001d00 tx=0x7f2aa4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:31.967+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2aac052e60 con 0x7f2aa8003a60 2023-10-06T11:42:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.161+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2abc0ac6c0 con 0x7f2aa8003a60 2023-10-06T11:42:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.162+0000 7f2ab27fc700 1 -- 172.21.15.16:0/2265833576 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f2aac02b090 con 0x7f2aa8003a60 2023-10-06T11:42:32.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:32.165 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 msgr2=0x7f2a9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 0x7f2a9c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4001d00 tx=0x7f2aa4009040 comp rx=0 tx=0).stop 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 msgr2=0x7f2abc14f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c0b0 tx=0x7f2aac01a000 comp rx=0 tx=0).stop 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 shutdown_connections 2023-10-06T11:42:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2aa8003a60 0x7f2abc14f470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2a9c066f30 0x7f2a9c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2abc0b17a0 0x7f2abc0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 --2- 172.21.15.16:0/2265833576 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2abc0abd10 0x7f2abc0ac180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 >> 172.21.15.16:0/2265833576 conn(0x7f2abc0cd480 msgr2=0x7f2abc06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 shutdown_connections 2023-10-06T11:42:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.164+0000 7f2ac0d51700 1 -- 172.21.15.16:0/2265833576 wait complete. 2023-10-06T11:42:32.179 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:42:32.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.305+0000 7f78124cd700 1 Processor -- start 2023-10-06T11:42:32.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.305+0000 7f78124cd700 1 -- start start 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78124cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c1593c0 0x7f780c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78124cd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c159790 0x7f780c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78124cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780c15a650 con 0x7f780c0b53f0 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78124cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780c0b5f60 con 0x7f780c1593c0 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78124cd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c0b61c0 con 0x7f780c159790 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c1593c0 0x7f780c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f780c1593c0 0x7f780c15d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37162/0 (socket says 172.21.15.16:37162) 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 -- 172.21.15.16:0/2441199066 learned_addr learned my addr 172.21.15.16:0/2441199066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f7811ccc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f780c0b53f0 0x7f780c0b5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38092/0 (socket says 172.21.15.16:38092) 2023-10-06T11:42:32.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f7810cca700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c159790 0x7f780c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 -- 172.21.15.16:0/2441199066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c159790 msgr2=0x7f780c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c159790 0x7f780c15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 -- 172.21.15.16:0/2441199066 >> v1:172.21.15.16:6789/0 conn(0x7f780c0b53f0 legacy=0x7f780c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f78114cb700 1 -- 172.21.15.16:0/2441199066 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c159d20 con 0x7f780c1593c0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.306+0000 7f7803fff700 1 -- 172.21.15.16:0/2441199066 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892697292 0 0) 0x7f780c15a650 con 0x7f780c0b53f0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78114cb700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c1593c0 0x7f780c15d210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7808009f70 tx=0x7f7808009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c874262961248f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f7803fff700 1 -- 172.21.15.16:0/2441199066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780800e070 con 0x7f780c1593c0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f7803fff700 1 -- 172.21.15.16:0/2441199066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780800ba20 con 0x7f780c1593c0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f7803fff700 1 -- 172.21.15.16:0/2441199066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080092e0 con 0x7f780c1593c0 2023-10-06T11:42:32.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 -- 172.21.15.16:0/2441199066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c1593c0 msgr2=0x7f780c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c1593c0 0x7f780c15d210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7808009f70 tx=0x7f7808009840 comp rx=0 tx=0).stop 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 -- 172.21.15.16:0/2441199066 shutdown_connections 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c1593c0 0x7f780c15d210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 --2- 172.21.15.16:0/2441199066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c159790 0x7f780c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 -- 172.21.15.16:0/2441199066 >> 172.21.15.16:0/2441199066 conn(0x7f780c08a020 msgr2=0x7f780c099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 -- 172.21.15.16:0/2441199066 shutdown_connections 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.307+0000 7f78124cd700 1 -- 172.21.15.16:0/2441199066 wait complete. 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.308+0000 7f78124cd700 1 Processor -- start 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.308+0000 7f78124cd700 1 -- start start 2023-10-06T11:42:32.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c1593c0 0x7f780c09b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780c159790 0x7f780c143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c09bc70 con 0x7f780c0b53f0 2023-10-06T11:42:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780c09a690 con 0x7f780c159790 2023-10-06T11:42:32.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78124cd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780c09a970 con 0x7f780c1593c0 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78114cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78114cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37172/0 (socket says 172.21.15.16:37172) 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.309+0000 7f78114cb700 1 -- 172.21.15.16:0/3211530959 learned_addr learned my addr 172.21.15.16:0/3211530959 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f7810cca700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c1593c0 0x7f780c09b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f7811ccc700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780c159790 0x7f780c143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 -- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c1593c0 msgr2=0x7f780c09b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c1593c0 0x7f780c09b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 -- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780c159790 msgr2=0x7f780c143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780c159790 0x7f780c143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 -- 172.21.15.16:0/3211530959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808009ae0 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78114cb700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f780c065950 tx=0x7f7808018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7808008be0 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7808008d60 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780c09ac50 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080266b0 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.310+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780c09b210 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.311+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780c09b430 con 0x7f780c0b53f0 2023-10-06T11:42:32.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.311+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7808003040 con 0x7f780c0b53f0 2023-10-06T11:42:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.312+0000 7f78017fa700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 0x7f77f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.312+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7808086580 con 0x7f780c0b53f0 2023-10-06T11:42:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.312+0000 7f7810cca700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 0x7f77f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.312+0000 7f7810cca700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 0x7f77f00693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f780c0b1d20 tx=0x7f77fc009120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.316+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7808053690 con 0x7f780c0b53f0 2023-10-06T11:42:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.557+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f780c04c380 con 0x7f780c0b53f0 2023-10-06T11:42:32.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.557+0000 7f78017fa700 1 -- 172.21.15.16:0/3211530959 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7f7808056cb0 con 0x7f780c0b53f0 2023-10-06T11:42:32.560 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:32.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 msgr2=0x7f77f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 0x7f77f00693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f780c0b1d20 tx=0x7f77fc009120 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 msgr2=0x7f780c15c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f780c065950 tx=0x7f7808018000 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 shutdown_connections 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f780c0b53f0 0x7f780c15c540 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f77f0066f30 0x7f77f00693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f780c159790 0x7f780c143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 --2- 172.21.15.16:0/3211530959 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f780c1593c0 0x7f780c09b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 >> 172.21.15.16:0/3211530959 conn(0x7f780c08a020 msgr2=0x7f780c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 shutdown_connections 2023-10-06T11:42:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.559+0000 7f78124cd700 1 -- 172.21.15.16:0/3211530959 wait complete. 2023-10-06T11:42:32.563 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:32.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.691+0000 7fd90edb6700 1 Processor -- start 2023-10-06T11:42:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.691+0000 7fd90edb6700 1 -- start start 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080b3110 con 0x7fd90815cc20 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080cdb10 con 0x7fd9080cd6a0 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90edb6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080ccb20 con 0x7fd9080cf280 2023-10-06T11:42:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90cb52700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90cb52700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37182/0 (socket says 172.21.15.16:37182) 2023-10-06T11:42:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.692+0000 7fd90cb52700 1 -- 172.21.15.16:0/3264938523 learned_addr learned my addr 172.21.15.16:0/3264938523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd907fff700 1 --2- 172.21.15.16:0/3264938523 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90d353700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 -- 172.21.15.16:0/3264938523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 msgr2=0x7fd90815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 -- 172.21.15.16:0/3264938523 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 msgr2=0x7fd9080ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 --2- 172.21.15.16:0/3264938523 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 -- 172.21.15.16:0/3264938523 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080cbf10 con 0x7fd90815cc20 2023-10-06T11:42:32.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd907fff700 1 --2- 172.21.15.16:0/3264938523 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd90cb52700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9080cb150 tx=0x7fd8f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16e7202ae47f598 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd9077fe700 1 -- 172.21.15.16:0/3264938523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800b860 con 0x7fd90815cc20 2023-10-06T11:42:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd9077fe700 1 -- 172.21.15.16:0/3264938523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800b9c0 con 0x7fd90815cc20 2023-10-06T11:42:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.693+0000 7fd9077fe700 1 -- 172.21.15.16:0/3264938523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800bd80 con 0x7fd90815cc20 2023-10-06T11:42:32.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 -- 172.21.15.16:0/3264938523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 msgr2=0x7fd9080cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9080cb150 tx=0x7fd8f8009250 comp rx=0 tx=0).stop 2023-10-06T11:42:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 -- 172.21.15.16:0/3264938523 shutdown_connections 2023-10-06T11:42:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd90815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 --2- 172.21.15.16:0/3264938523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90815cc20 0x7fd9080cd160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 --2- 172.21.15.16:0/3264938523 >> v2:172.21.15.177:3300/0 conn(0x7fd9080cd6a0 0x7fd9080ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 -- 172.21.15.16:0/3264938523 >> 172.21.15.16:0/3264938523 conn(0x7fd908099b40 msgr2=0x7fd90806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 -- 172.21.15.16:0/3264938523 shutdown_connections 2023-10-06T11:42:32.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.694+0000 7fd90edb6700 1 -- 172.21.15.16:0/3264938523 wait complete. 2023-10-06T11:42:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.695+0000 7fd90edb6700 1 Processor -- start 2023-10-06T11:42:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.695+0000 7fd90edb6700 1 -- start start 2023-10-06T11:42:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd9081590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90809d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908159620 con 0x7fd9080cd6a0 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c8ef0 con 0x7fd90815cc20 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd908169720 con 0x7fd9080cf280 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37194/0 (socket says 172.21.15.16:37194) 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 -- 172.21.15.16:0/2281352800 learned_addr learned my addr 172.21.15.16:0/2281352800 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd907fff700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd9081590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90d353700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90809d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 -- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 msgr2=0x7fd9081590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd9081590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 -- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 msgr2=0x7fd90809d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90809d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 -- 172.21.15.16:0/2281352800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f80094f0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd907fff700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd9081590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90d353700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90809d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90cb52700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd908089390 tx=0x7fd8f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8026450 con 0x7fd9080cd6a0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80265b0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90809dd30 con 0x7fd9080cd6a0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.696+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8026970 con 0x7fd9080cd6a0 2023-10-06T11:42:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.697+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9080c8b80 con 0x7fd9080cd6a0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.698+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90804c380 con 0x7fd9080cd6a0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.698+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8f8026ce0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.698+0000 7fd904ff9700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 0x7fd8f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.698+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd8f8086130 con 0x7fd9080cd6a0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.699+0000 7fd907fff700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 0x7fd8f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.699+0000 7fd907fff700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 0x7fd8f40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006f00 tx=0x7fd8fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:32.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.703+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8f80532c0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.940+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9080cc1d0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.940+0000 7fd904ff9700 1 -- 172.21.15.16:0/2281352800 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4591 (secure 0 0 0) 0x7fd8f80568e0 con 0x7fd9080cd6a0 2023-10-06T11:42:32.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:32.943 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/1823271857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":1823271857},{"type":"v1","addr":"172.21.15.177:6836","nonce":1823271857}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/545571190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":545571190},{"type":"v1","addr":"172.21.15.16:6834","nonce":545571190}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/299311337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":299311337},{"type":"v1","addr":"172.21.15.16:6835","nonce":299311337}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:41:55.893171+0000","modified":"2023-10-06T11:41:59.529047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6837/1602610299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":1602610299},{"type":"v1","addr":"172.21.15.177:6837","nonce":1602610299}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T11:42:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 msgr2=0x7fd8f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 0x7fd8f40693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc006f00 tx=0x7fd8fc008040 comp rx=0 tx=0).stop 2023-10-06T11:42:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 msgr2=0x7fd9080c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd908089390 tx=0x7fd8f8017000 comp rx=0 tx=0).stop 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 shutdown_connections 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9080cd6a0 0x7fd9080c82d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8f4066f30 0x7fd8f40693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd90815cc20 0x7fd90809d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 --2- 172.21.15.16:0/2281352800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9080cf280 0x7fd9081590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 >> 172.21.15.16:0/2281352800 conn(0x7fd908099b40 msgr2=0x7fd9080c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 shutdown_connections 2023-10-06T11:42:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:32.942+0000 7fd90edb6700 1 -- 172.21.15.16:0/2281352800 wait complete. 2023-10-06T11:42:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-10-06T11:42:32.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.074+0000 7f59eb59e700 1 Processor -- start 2023-10-06T11:42:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.074+0000 7f59eb59e700 1 -- start start 2023-10-06T11:42:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59eb59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59ec15d920 0x7f59ec0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59eb59e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec15a650 con 0x7f59ec1593c0 2023-10-06T11:42:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59eb59e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0b61c0 con 0x7f59ec159790 2023-10-06T11:42:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59eb59e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec159d20 con 0x7f59ec15d920 2023-10-06T11:42:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59e9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59ec15d920 0x7f59ec0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59ea59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f59ec1593c0 0x7f59ec15d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38096/0 (socket says 172.21.15.16:38096) 2023-10-06T11:42:33.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59ea59c700 1 -- 172.21.15.16:0/3395535845 learned_addr learned my addr 172.21.15.16:0/3395535845 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3395535845 >> v1:172.21.15.177:6789/0 conn(0x7f59ec159790 legacy=0x7f59ec0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3395535845 >> v1:172.21.15.16:6789/0 conn(0x7f59ec1593c0 legacy=0x7f59ec15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.075+0000 7f59e959a700 1 -- 172.21.15.16:0/3395535845 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602039037 0 0) 0x7f59ec15a650 con 0x7f59ec1593c0 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3395535845 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec15a000 con 0x7f59ec15d920 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59e9d9b700 1 --2- 172.21.15.16:0/3395535845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec15d920 0x7f59ec0b53f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59d40098d0 tx=0x7f59d4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b3b7391ab57c77 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59e959a700 1 -- 172.21.15.16:0/3395535845 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4009e70 con 0x7f59ec15d920 2023-10-06T11:42:33.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59e959a700 1 -- 172.21.15.16:0/3395535845 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4005c10 con 0x7f59ec15d920 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59e959a700 1 -- 172.21.15.16:0/3395535845 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d400b610 con 0x7f59ec15d920 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 -- 172.21.15.16:0/3395535845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec15d920 msgr2=0x7f59ec0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3395535845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec15d920 0x7f59ec0b53f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f59d40098d0 tx=0x7f59d4009120 comp rx=0 tx=0).stop 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 -- 172.21.15.16:0/3395535845 shutdown_connections 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3395535845 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec15d920 0x7f59ec0b53f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 -- 172.21.15.16:0/3395535845 >> 172.21.15.16:0/3395535845 conn(0x7f59ec08a020 msgr2=0x7f59ec099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 -- 172.21.15.16:0/3395535845 shutdown_connections 2023-10-06T11:42:33.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.076+0000 7f59eb59e700 1 -- 172.21.15.16:0/3395535845 wait complete. 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.077+0000 7f59eb59e700 1 Processor -- start 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 -- start start 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59ec1593c0 0x7f59ec151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59ec15d920 0x7f59ec15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec142790 con 0x7f59ec159790 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec142c20 con 0x7f59ec15d920 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59eb59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec141b80 con 0x7f59ec1593c0 2023-10-06T11:42:33.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59e9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59e9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37206/0 (socket says 172.21.15.16:37206) 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.078+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3976988473 learned_addr learned my addr 172.21.15.16:0/3976988473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59ec1593c0 msgr2=0x7f59ec151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59ec1593c0 0x7f59ec151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59ec15d920 msgr2=0x7f59ec15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59ead9d700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59ec15d920 0x7f59ec15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59ec15d920 0x7f59ec15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 -- 172.21.15.16:0/3976988473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40093c0 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e9d9b700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009160 tx=0x7f59d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40066c0 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4006820 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec141e60 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4026680 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.079+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec142420 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.080+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec142a70 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.080+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59d4023070 con 0x7f59ec159790 2023-10-06T11:42:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.081+0000 7f59e27fc700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 0x7f59cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.081+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f59d4086b00 con 0x7f59ec159790 2023-10-06T11:42:33.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.081+0000 7f59ea59c700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 0x7f59cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.082+0000 7f59ea59c700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 0x7f59cc0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b1410 tx=0x7f59e4007810 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.085+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59d4053c10 con 0x7f59ec159790 2023-10-06T11:42:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.278+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59ec1428d0 con 0x7f59ec159790 2023-10-06T11:42:33.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.279+0000 7f59e27fc700 1 -- 172.21.15.16:0/3976988473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f59ec1428d0 con 0x7f59ec159790 2023-10-06T11:42:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:33.281 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:41:57.495140+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:41:53.661288+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T11:41:54.874734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:33.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 msgr2=0x7f59cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 0x7f59cc0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b1410 tx=0x7f59e4007810 comp rx=0 tx=0).stop 2023-10-06T11:42:33.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 msgr2=0x7f59ec15e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009160 tx=0x7f59d4015000 comp rx=0 tx=0).stop 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 shutdown_connections 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59cc066f30 0x7f59cc0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59ec15d920 0x7f59ec15e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f59ec1593c0 0x7f59ec151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 --2- 172.21.15.16:0/3976988473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59ec159790 0x7f59ec15e370 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 >> 172.21.15.16:0/3976988473 conn(0x7f59ec08a020 msgr2=0x7f59ec152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 shutdown_connections 2023-10-06T11:42:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.281+0000 7f59eb59e700 1 -- 172.21.15.16:0/3976988473 wait complete. 2023-10-06T11:42:33.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:42:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.413+0000 7ff93cf2b700 1 Processor -- start 2023-10-06T11:42:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.413+0000 7ff93cf2b700 1 -- start start 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff93cf2b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff9380ce1e0 0x7ff93806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff93cf2b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938159240 0x7ff938159680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff93cf2b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380c88e0 con 0x7ff93806c5d0 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff93cf2b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380c8d70 con 0x7ff938159240 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff93cf2b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff938159bc0 con 0x7ff9380ce1e0 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff936d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938159240 0x7ff938159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff936d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938159240 0x7ff938159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37220/0 (socket says 172.21.15.16:37220) 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff936d9d700 1 -- 172.21.15.16:0/3361480747 learned_addr learned my addr 172.21.15.16:0/3361480747 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.414+0000 7ff935d9b700 1 --2- 172.21.15.16:0/3361480747 >> v2:172.21.15.177:3300/0 conn(0x7ff9380ce1e0 0x7ff93806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff936d9d700 1 -- 172.21.15.16:0/3361480747 >> v2:172.21.15.177:3300/0 conn(0x7ff9380ce1e0 msgr2=0x7ff93806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff936d9d700 1 --2- 172.21.15.16:0/3361480747 >> v2:172.21.15.177:3300/0 conn(0x7ff9380ce1e0 0x7ff93806c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff936d9d700 1 -- 172.21.15.16:0/3361480747 >> v1:172.21.15.177:6789/0 conn(0x7ff93806c5d0 legacy=0x7ff9380cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff936d9d700 1 -- 172.21.15.16:0/3361480747 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380c7cd0 con 0x7ff938159240 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff936d9d700 1 --2- 172.21.15.16:0/3361480747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938159240 0x7ff938159680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f20 tx=0x7ff930009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2f678ed3b278d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff93559a700 1 -- 172.21.15.16:0/3361480747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000e070 con 0x7ff938159240 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff93559a700 1 -- 172.21.15.16:0/3361480747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000b990 con 0x7ff938159240 2023-10-06T11:42:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff93559a700 1 -- 172.21.15.16:0/3361480747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300092e0 con 0x7ff938159240 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff93cf2b700 1 -- 172.21.15.16:0/3361480747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938159240 msgr2=0x7ff938159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.415+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/3361480747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938159240 0x7ff938159680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff930009f20 tx=0x7ff930009840 comp rx=0 tx=0).stop 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 -- 172.21.15.16:0/3361480747 shutdown_connections 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/3361480747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938159240 0x7ff938159680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/3361480747 >> v2:172.21.15.177:3300/0 conn(0x7ff9380ce1e0 0x7ff93806c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 -- 172.21.15.16:0/3361480747 >> 172.21.15.16:0/3361480747 conn(0x7ff93806d520 msgr2=0x7ff93806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 -- 172.21.15.16:0/3361480747 shutdown_connections 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.416+0000 7ff93cf2b700 1 -- 172.21.15.16:0/3361480747 wait complete. 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 Processor -- start 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 -- start start 2023-10-06T11:42:33.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9380ce1e0 0x7ff938152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff938159240 0x7ff93806aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815f300 con 0x7ff93806c5d0 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9381538b0 con 0x7ff9380ce1e0 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.417+0000 7ff93cf2b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93816d140 con 0x7ff938159240 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37226/0 (socket says 172.21.15.16:37226) 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 -- 172.21.15.16:0/1717237063 learned_addr learned my addr 172.21.15.16:0/1717237063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff935d9b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9380ce1e0 0x7ff938152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff936d9d700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff938159240 0x7ff93806aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 -- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff938159240 msgr2=0x7ff93806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff938159240 0x7ff93806aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 -- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9380ce1e0 msgr2=0x7ff938152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9380ce1e0 0x7ff938152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 -- 172.21.15.16:0/1717237063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930009ae0 con 0x7ff93806c5d0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93659c700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7c0 tx=0x7ff928007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011590 con 0x7ff93806c5d0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9280116f0 con 0x7ff93806c5d0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93816d420 con 0x7ff93806c5d0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011ab0 con 0x7ff93806c5d0 2023-10-06T11:42:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.418+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93816d9e0 con 0x7ff93806c5d0 2023-10-06T11:42:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.419+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938063e70 con 0x7ff93806c5d0 2023-10-06T11:42:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.420+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff92801b3b0 con 0x7ff93806c5d0 2023-10-06T11:42:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.420+0000 7ff92e7fc700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 0x7ff9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.420+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff928083a10 con 0x7ff93806c5d0 2023-10-06T11:42:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.420+0000 7ff935d9b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 0x7ff9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:33.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.421+0000 7ff935d9b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 0x7ff9180693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff938155090 tx=0x7ff9200091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:33.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.424+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff928050bd0 con 0x7ff93806c5d0 2023-10-06T11:42:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.661+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff9380c7fb0 con 0x7ff93806c5d0 2023-10-06T11:42:34.330 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-10-06T11:42:33.662+0000 7f92ac3a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:42:34.331 INFO:tasks.ceph.mds.b.smithi177.stderr: -3376> 2023-10-06T11:41:50.558+0000 7f65a9010700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-06T11:42:34.331 INFO:tasks.ceph.mds.b.smithi177.stderr: -2160> 2023-10-06T11:42:01.567+0000 7f65a3004700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:42:34.331 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.665+0000 7ff92e7fc700 1 -- 172.21.15.16:0/1717237063 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff928017070 con 0x7ff93806c5d0 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 msgr2=0x7ff9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 0x7ff9180693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff938155090 tx=0x7ff9200091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 msgr2=0x7ff93815edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c7c0 tx=0x7ff928007f70 comp rx=0 tx=0).stop 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 shutdown_connections 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff93806c5d0 0x7ff93815edc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff918066f30 0x7ff9180693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff938159240 0x7ff93806aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 --2- 172.21.15.16:0/1717237063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9380ce1e0 0x7ff938152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 >> 172.21.15.16:0/1717237063 conn(0x7ff93806d520 msgr2=0x7ff9380cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 shutdown_connections 2023-10-06T11:42:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:33.667+0000 7ff93cf2b700 1 -- 172.21.15.16:0/1717237063 wait complete. 2023-10-06T11:42:34.334 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:42:34.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:42:34.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.463+0000 7f9566bc0700 1 Processor -- start 2023-10-06T11:42:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.463+0000 7f9566bc0700 1 -- start start 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f9566bc0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f95600cda10 0x7f95600b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f9566bc0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600cead0 con 0x7f9560159b10 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f9566bc0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600affc0 con 0x7f9560159740 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f9566bc0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600cdec0 con 0x7f95600cda10 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f956515d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9560159b10 0x7f95600cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38108/0 (socket says 172.21.15.16:38108) 2023-10-06T11:42:34.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f956515d700 1 -- 172.21.15.16:0/1647452788 learned_addr learned my addr 172.21.15.16:0/1647452788 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.464+0000 7f955ffff700 1 --2- 172.21.15.16:0/1647452788 >> v2:172.21.15.177:3300/0 conn(0x7f95600cda10 0x7f95600b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819055994 0 0) 0x7f95600cead0 con 0x7f9560159b10 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9548003570 con 0x7f9560159b10 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3926175285 0 0) 0x7f9548003570 con 0x7f9560159b10 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 >> v2:172.21.15.177:3300/0 conn(0x7f95600cda10 msgr2=0x7f95600b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 --2- 172.21.15.16:0/1647452788 >> v2:172.21.15.177:3300/0 conn(0x7f95600cda10 0x7f95600b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 >> v1:172.21.15.177:6789/0 conn(0x7f9560159740 legacy=0x7f95600b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600ce1a0 con 0x7f9560159b10 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f95580017f0 con 0x7f9560159b10 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 >> v1:172.21.15.16:6789/0 conn(0x7f9560159b10 legacy=0x7f95600cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 0x7f9548003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9548004810 0x7f9548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95480051c0 0x7f954800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cdec0 con 0x7f9548003a60 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954800c160 con 0x7f95480051c0 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954800c320 con 0x7f9548004810 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 0x7f9548003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956515d700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95480051c0 0x7f954800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955ffff700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9548004810 0x7f9548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 -- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9548004810 msgr2=0x7f9548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9548004810 0x7f9548004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 -- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95480051c0 msgr2=0x7f954800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95480051c0 0x7f954800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 -- 172.21.15.16:0/1647452788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548003570 con 0x7f9548003a60 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956515d700 1 -- 172.21.15.16:0/1647452788 reap_dead start 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f955ffff700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9548004810 0x7f9548004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.465+0000 7f956495c700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 0x7f9548003e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95500018e0 tx=0x7f9550007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154370c72834c5fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550012420 con 0x7f9548003a60 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95500125a0 con 0x7f9548003a60 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f955f7fe700 1 -- 172.21.15.16:0/1647452788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550012960 con 0x7f9548003a60 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 -- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 msgr2=0x7f9548003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 0x7f9548003e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95500018e0 tx=0x7f9550007d30 comp rx=0 tx=0).stop 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 -- 172.21.15.16:0/1647452788 shutdown_connections 2023-10-06T11:42:34.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 --2- 172.21.15.16:0/1647452788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9548003a60 0x7f9548003e70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 -- 172.21.15.16:0/1647452788 >> 172.21.15.16:0/1647452788 conn(0x7f9560099a40 msgr2=0x7f956006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 -- 172.21.15.16:0/1647452788 shutdown_connections 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.466+0000 7f9566bc0700 1 -- 172.21.15.16:0/1647452788 wait complete. 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.467+0000 7f9566bc0700 1 Processor -- start 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 -- start start 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9548003a60 0x7f9560156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f956009dce0 0x7f956006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956016ab50 con 0x7f956015ee00 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956016a240 con 0x7f9548003a60 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f9566bc0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956009e180 con 0x7f956009dce0 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f955ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f955ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37250/0 (socket says 172.21.15.16:37250) 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f955ffff700 1 -- 172.21.15.16:0/834530443 learned_addr learned my addr 172.21.15.16:0/834530443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f956495c700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9548003a60 0x7f9560156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.468+0000 7f956515d700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f956009dce0 0x7f956006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 -- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f956009dce0 msgr2=0x7f956006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f956009dce0 0x7f956006eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 -- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9548003a60 msgr2=0x7f9560156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9548003a60 0x7f9560156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:34.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 -- 172.21.15.16:0/834530443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956016a500 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955ffff700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9554011700 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554015670 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95540157d0 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956016a7e0 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554015b90 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.469+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9560156ea0 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.470+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600b6540 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.470+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9554013410 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.471+0000 7f955cff9700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 0x7f954c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.471+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f95540848e0 con 0x7f956015ee00 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.471+0000 7f956495c700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 0x7f954c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:34.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.471+0000 7f956495c700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 0x7f954c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95500016c0 tx=0x7f9550015000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.475+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95540519a0 con 0x7f956015ee00 2023-10-06T11:42:34.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:34.717+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f956004c380 con 0x7f956015ee00 2023-10-06T11:42:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.670+0000 7f955cff9700 1 -- 172.21.15.16:0/834530443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9554054fc0 con 0x7f956015ee00 2023-10-06T11:42:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 msgr2=0x7f954c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 0x7f954c0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f95500016c0 tx=0x7f9550015000 comp rx=0 tx=0).stop 2023-10-06T11:42:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 msgr2=0x7f956009d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9554011700 tx=0x7f9554008040 comp rx=0 tx=0).stop 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 shutdown_connections 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f954c066f30 0x7f954c0693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f956009dce0 0x7f956006eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9548003a60 0x7f9560156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 --2- 172.21.15.16:0/834530443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956015ee00 0x7f956009d7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 >> 172.21.15.16:0/834530443 conn(0x7f9560099a40 msgr2=0x7f95600cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 shutdown_connections 2023-10-06T11:42:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.673+0000 7f9566bc0700 1 -- 172.21.15.16:0/834530443 wait complete. 2023-10-06T11:42:35.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:42:35.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.818+0000 7fb68f59e700 1 Processor -- start 2023-10-06T11:42:35.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.818+0000 7fb68f59e700 1 -- start start 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68f59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6900ccff0 0x7fb6900b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68f59e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6900b33b0 con 0x7fb6900b4850 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68f59e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb690159a70 con 0x7fb6900d04e0 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68f59e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6900cd450 con 0x7fb6900ccff0 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6900ccff0 0x7fb6900b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6900b4850 0x7fb690161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38118/0 (socket says 172.21.15.16:38118) 2023-10-06T11:42:35.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68dd9b700 1 -- 172.21.15.16:0/791055488 learned_addr learned my addr 172.21.15.16:0/791055488 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68ed9d700 1 -- 172.21.15.16:0/791055488 >> v1:172.21.15.177:6789/0 conn(0x7fb6900d04e0 legacy=0x7fb6900d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68ed9d700 1 -- 172.21.15.16:0/791055488 >> v1:172.21.15.16:6789/0 conn(0x7fb6900b4850 legacy=0x7fb690161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68ed9d700 1 -- 172.21.15.16:0/791055488 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6900cf800 con 0x7fb6900ccff0 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.819+0000 7fb68d59a700 1 -- 172.21.15.16:0/791055488 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419141762 0 0) 0x7fb6900b33b0 con 0x7fb6900b4850 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68ed9d700 1 --2- 172.21.15.16:0/791055488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900ccff0 0x7fb6900b6030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e60 tx=0x7fb680009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6caf46691f3285 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68d59a700 1 -- 172.21.15.16:0/791055488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68000e070 con 0x7fb6900ccff0 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68d59a700 1 -- 172.21.15.16:0/791055488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68000bbf0 con 0x7fb6900ccff0 2023-10-06T11:42:35.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68d59a700 1 -- 172.21.15.16:0/791055488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680012430 con 0x7fb6900ccff0 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 -- 172.21.15.16:0/791055488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900ccff0 msgr2=0x7fb6900b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 --2- 172.21.15.16:0/791055488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900ccff0 0x7fb6900b6030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e60 tx=0x7fb680009780 comp rx=0 tx=0).stop 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 -- 172.21.15.16:0/791055488 shutdown_connections 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 --2- 172.21.15.16:0/791055488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900ccff0 0x7fb6900b6030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 -- 172.21.15.16:0/791055488 >> 172.21.15.16:0/791055488 conn(0x7fb69006cea0 msgr2=0x7fb69006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 -- 172.21.15.16:0/791055488 shutdown_connections 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.820+0000 7fb68f59e700 1 -- 172.21.15.16:0/791055488 wait complete. 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 Processor -- start 2023-10-06T11:42:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 -- start start 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6900ccff0 0x7fb690156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6900d04e0 0x7fb6901574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6901698e0 con 0x7fb6900b4850 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900c5a80 con 0x7fb6900d04e0 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68f59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6900b2a20 con 0x7fb6900ccff0 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.822+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37272/0 (socket says 172.21.15.16:37272) 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 -- 172.21.15.16:0/3844593643 learned_addr learned my addr 172.21.15.16:0/3844593643 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68ed9d700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6900d04e0 0x7fb6901574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68dd9b700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6900ccff0 0x7fb690156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 -- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6900ccff0 msgr2=0x7fb690156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6900ccff0 0x7fb690156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 -- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6900d04e0 msgr2=0x7fb6901574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6900d04e0 0x7fb6901574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 -- 172.21.15.16:0/3844593643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680009a20 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68e59c700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb68800b5e0 tx=0x7fb688009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800f510 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68800f670 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6900c5430 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68800fa30 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.823+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6900b2d00 con 0x7fb6900b4850 2023-10-06T11:42:35.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.824+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69004c380 con 0x7fb6900b4850 2023-10-06T11:42:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.824+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb68801a400 con 0x7fb6900b4850 2023-10-06T11:42:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.825+0000 7fb6867fc700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 0x7fb6700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.825+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb688084210 con 0x7fb6900b4850 2023-10-06T11:42:35.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.825+0000 7fb68dd9b700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 0x7fb6700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.826+0000 7fb68dd9b700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 0x7fb6700693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb69015b330 tx=0x7fb678009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:35.829+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb688051350 con 0x7fb6900b4850 2023-10-06T11:42:36.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.020+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb69015b8d0 con 0x7fb6900b4850 2023-10-06T11:42:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.678+0000 7fb6867fc700 1 -- 172.21.15.16:0/3844593643 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb688054970 con 0x7fb6900b4850 2023-10-06T11:42:36.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 msgr2=0x7fb6700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 0x7fb6700693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb69015b330 tx=0x7fb678009040 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 msgr2=0x7fb690155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb68800b5e0 tx=0x7fb688009120 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 shutdown_connections 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6900b4850 0x7fb690155280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb670066f30 0x7fb6700693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb6900d04e0 0x7fb6901574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 --2- 172.21.15.16:0/3844593643 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb6900ccff0 0x7fb690156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 >> 172.21.15.16:0/3844593643 conn(0x7fb69006cea0 msgr2=0x7fb6900cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 shutdown_connections 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.682+0000 7fb68f59e700 1 -- 172.21.15.16:0/3844593643 wait complete. 2023-10-06T11:42:36.699 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:42:36.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:42:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.823+0000 7f2a04c37700 1 Processor -- start 2023-10-06T11:42:36.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.823+0000 7f2a04c37700 1 -- start start 2023-10-06T11:42:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f2a04c37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a000b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f2a04c37700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a000b0fe0 0x7f2a000b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f2a04c37700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a000b4b00 con 0x7f2a000b46a0 2023-10-06T11:42:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f2a04c37700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a000b2150 con 0x7f2a000b0fe0 2023-10-06T11:42:36.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f2a04c37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a000b1540 con 0x7f2a00159650 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2a000b0fe0 0x7f2a000b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2a000b46a0 0x7f2a00159a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fed9d700 1 -- 172.21.15.16:0/287109683 learned_addr learned my addr 172.21.15.16:0/287109683 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 -- 172.21.15.16:0/287109683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 msgr2=0x7f2a000b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 --2- 172.21.15.16:0/287109683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a000b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 -- 172.21.15.16:0/287109683 >> v1:172.21.15.16:6789/0 conn(0x7f2a000b46a0 legacy=0x7f2a00159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 -- 172.21.15.16:0/287109683 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a000b1820 con 0x7f2a000b0fe0 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fdd9b700 1 --2- 172.21.15.16:0/287109683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a000b4160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29e80098d0 tx=0x7f29e8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af50631ecff844fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fd59a700 1 -- 172.21.15.16:0/287109683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8009e70 con 0x7f2a000b0fe0 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fd59a700 1 -- 172.21.15.16:0/287109683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8005c10 con 0x7f2a000b0fe0 2023-10-06T11:42:36.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.824+0000 7f29fd59a700 1 -- 172.21.15.16:0/287109683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e800b650 con 0x7f2a000b0fe0 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 -- 172.21.15.16:0/287109683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 msgr2=0x7f2a000b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 --2- 172.21.15.16:0/287109683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a000b4160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f29e80098d0 tx=0x7f29e8009120 comp rx=0 tx=0).stop 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 -- 172.21.15.16:0/287109683 shutdown_connections 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 --2- 172.21.15.16:0/287109683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a000b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 --2- 172.21.15.16:0/287109683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a000b4160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 -- 172.21.15.16:0/287109683 >> 172.21.15.16:0/287109683 conn(0x7f2a0008a1e0 msgr2=0x7f2a00099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 -- 172.21.15.16:0/287109683 shutdown_connections 2023-10-06T11:42:36.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.825+0000 7f2a04c37700 1 -- 172.21.15.16:0/287109683 wait complete. 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.826+0000 7f2a04c37700 1 Processor -- start 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.826+0000 7f2a04c37700 1 -- start start 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a000b46a0 0x7f2a00148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a0009d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0016cd40 con 0x7f2a000b0fe0 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0009d960 con 0x7f2a000b46a0 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f2a04c37700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0009dc40 con 0x7f2a00159650 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37284/0 (socket says 172.21.15.16:37284) 2023-10-06T11:42:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 -- 172.21.15.16:0/3759174952 learned_addr learned my addr 172.21.15.16:0/3759174952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 -- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 msgr2=0x7f2a0009d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a0009d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 -- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a000b46a0 msgr2=0x7f2a00148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fdd9b700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a000b46a0 0x7f2a00148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a000b46a0 0x7f2a00148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 -- 172.21.15.16:0/3759174952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80093c0 con 0x7f2a000b0fe0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.827+0000 7f29fe59c700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f29f800b810 tx=0x7f29f80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.828+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f800f570 con 0x7f2a000b0fe0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.828+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f800f6d0 con 0x7f2a000b0fe0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.828+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f800fa90 con 0x7f2a000b0fe0 2023-10-06T11:42:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.828+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0016cf90 con 0x7f2a000b0fe0 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.828+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a00149030 con 0x7f2a000b0fe0 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.829+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0009de00 con 0x7f2a000b0fe0 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.829+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29f801a460 con 0x7f2a000b0fe0 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.829+0000 7f29f67fc700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 0x7f29e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.829+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f29f8084210 con 0x7f2a000b0fe0 2023-10-06T11:42:36.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.831+0000 7f29fdd9b700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 0x7f29e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.834+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29f8051500 con 0x7f2a000b0fe0 2023-10-06T11:42:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:36.834+0000 7f29fdd9b700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 0x7f29e00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009160 tx=0x7f29e8016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.028+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a00063e70 con 0x7f2a000b0fe0 2023-10-06T11:42:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.681+0000 7f29f67fc700 1 -- 172.21.15.16:0/3759174952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29f8054b20 con 0x7f2a000b0fe0 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 msgr2=0x7f29e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 0x7f29e00693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009160 tx=0x7f29e8016000 comp rx=0 tx=0).stop 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 msgr2=0x7f2a0016d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f29f800b810 tx=0x7f29f80091d0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 shutdown_connections 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2a000b0fe0 0x7f2a0016d790 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f29e0066f30 0x7f29e00693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2a00159650 0x7f2a0009d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 --2- 172.21.15.16:0/3759174952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a000b46a0 0x7f2a00148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 >> 172.21.15.16:0/3759174952 conn(0x7f2a0008a1e0 msgr2=0x7f2a00158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 shutdown_connections 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.685+0000 7f2a04c37700 1 -- 172.21.15.16:0/3759174952 wait complete. 2023-10-06T11:42:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-10-06T11:42:37.702 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T11:42:37.702 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T11:42:37.702 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:42:37.702 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:42:37.703 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T11:42:37.703 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T11:42:37.703 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:42:37.703 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:42:37.703 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T11:42:37.703 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T11:42:37.704 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:42:37.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:42:37.704 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T11:42:37.704 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T11:42:37.704 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:42:37.704 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:42:37.715 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:42:37.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T11:42:37.716 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:42:37.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T11:42:37.717 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:42:37.717 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T11:42:37.718 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:42:37.718 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T11:42:37.757 INFO:tasks.ceph.mds.a:Started 2023-10-06T11:42:37.757 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:37.758 INFO:tasks.ceph.mds.c:Started 2023-10-06T11:42:37.759 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:37.759 INFO:tasks.ceph.mds.b:Started 2023-10-06T11:42:37.760 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:37.760 INFO:tasks.ceph.mds.d:Started 2023-10-06T11:42:37.760 DEBUG:teuthology.parallel:result is None 2023-10-06T11:42:37.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T11:42:37.836 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-10-06T11:42:37.837 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-10-06T11:42:37.839 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T11:42:37.839 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T11:42:37.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.886+0000 7f9e9d301700 1 Processor -- start 2023-10-06T11:42:37.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.886+0000 7f9e9d301700 1 -- start start 2023-10-06T11:42:37.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e9d301700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9815d920 0x7f9e980b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e9d301700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e98159790 0x7f9e980b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e9d301700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9815a650 con 0x7f9e981593c0 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e9d301700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e980b61c0 con 0x7f9e9815d920 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e9d301700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e98159d20 con 0x7f9e98159790 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e977fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9815d920 0x7f9e980b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e977fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e9815d920 0x7f9e980b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37322/0 (socket says 172.21.15.16:37322) 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.887+0000 7f9e977fe700 1 -- 172.21.15.16:0/787796965 learned_addr learned my addr 172.21.15.16:0/787796965 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9cb00700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e98159790 0x7f9e980b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e977fe700 1 -- 172.21.15.16:0/787796965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e98159790 msgr2=0x7f9e980b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:37.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e977fe700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e98159790 0x7f9e980b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e977fe700 1 -- 172.21.15.16:0/787796965 >> v1:172.21.15.177:6789/0 conn(0x7f9e981593c0 legacy=0x7f9e9815d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e977fe700 1 -- 172.21.15.16:0/787796965 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9815a000 con 0x7f9e9815d920 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e977fe700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9815d920 0x7f9e980b53f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009990 tx=0x7f9e8c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5caee03e493948 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e96ffd700 1 -- 172.21.15.16:0/787796965 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c005c10 con 0x7f9e9815d920 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e96ffd700 1 -- 172.21.15.16:0/787796965 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c005d70 con 0x7f9e9815d920 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e96ffd700 1 -- 172.21.15.16:0/787796965 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00b840 con 0x7f9e9815d920 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 -- 172.21.15.16:0/787796965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9815d920 msgr2=0x7f9e980b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9815d920 0x7f9e980b53f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009990 tx=0x7f9e8c009220 comp rx=0 tx=0).stop 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 -- 172.21.15.16:0/787796965 shutdown_connections 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e98159790 0x7f9e980b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 --2- 172.21.15.16:0/787796965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e9815d920 0x7f9e980b53f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.888+0000 7f9e9d301700 1 -- 172.21.15.16:0/787796965 >> 172.21.15.16:0/787796965 conn(0x7f9e9808a020 msgr2=0x7f9e98099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.889+0000 7f9e9d301700 1 -- 172.21.15.16:0/787796965 shutdown_connections 2023-10-06T11:42:37.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.889+0000 7f9e9d301700 1 -- 172.21.15.16:0/787796965 wait complete. 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.890+0000 7f9e9d301700 1 Processor -- start 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.890+0000 7f9e9d301700 1 -- start start 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.890+0000 7f9e9d301700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e981593c0 0x7f9e9814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.890+0000 7f9e9d301700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.890+0000 7f9e9d301700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e9815d920 0x7f9e9809b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9d301700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9809c830 con 0x7f9e98159790 2023-10-06T11:42:37.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9d301700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9814e100 con 0x7f9e981593c0 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9d301700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9809bc20 con 0x7f9e9815d920 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37332/0 (socket says 172.21.15.16:37332) 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 -- 172.21.15.16:0/3195751901 learned_addr learned my addr 172.21.15.16:0/3195751901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e97fff700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e981593c0 0x7f9e9814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9cb00700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e9815d920 0x7f9e9809b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 -- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e9815d920 msgr2=0x7f9e9809b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e9815d920 0x7f9e9809b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 -- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e981593c0 msgr2=0x7f9e9814cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e981593c0 0x7f9e9814cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 -- 172.21.15.16:0/3195751901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c009480 con 0x7f9e98159790 2023-10-06T11:42:37.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e977fe700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0030c0 tx=0x7f9e8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c000cf0 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c011b60 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c01c420 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9809bf00 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.891+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9809c4c0 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.892+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9809c9a0 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.893+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e8c01c810 con 0x7f9e98159790 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.893+0000 7f9e7ffff700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 0x7f9e840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:37.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.893+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9e8c085e30 con 0x7f9e98159790 2023-10-06T11:42:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.893+0000 7f9e97fff700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 0x7f9e840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:37.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.894+0000 7f9e97fff700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 0x7f9e840693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e88003a70 tx=0x7f9e88007ef0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:37.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:37.898+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e8c053240 con 0x7f9e98159790 2023-10-06T11:42:38.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.106+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9e9804c380 con 0x7f9e98159790 2023-10-06T11:42:38.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.699+0000 7f9e7ffff700 1 -- 172.21.15.16:0/3195751901 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e8c056860 con 0x7f9e98159790 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 msgr2=0x7f9e840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 0x7f9e840693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e88003a70 tx=0x7f9e88007ef0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 msgr2=0x7f9e9814dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0030c0 tx=0x7f9e8c015000 comp rx=0 tx=0).stop 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 shutdown_connections 2023-10-06T11:42:38.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e84066f30 0x7f9e840693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e9815d920 0x7f9e9809b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e981593c0 0x7f9e9814cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 --2- 172.21.15.16:0/3195751901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e98159790 0x7f9e9814dbc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 >> 172.21.15.16:0/3195751901 conn(0x7f9e9808a020 msgr2=0x7f9e9815d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 shutdown_connections 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.702+0000 7f9e9d301700 1 -- 172.21.15.16:0/3195751901 wait complete. 2023-10-06T11:42:38.705 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-10-06T11:42:38.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T11:42:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.838+0000 7ff2873dd700 1 Processor -- start 2023-10-06T11:42:38.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.838+0000 7ff2873dd700 1 -- start start 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff2873dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff28015cc20 0x7ff2800cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff2873dd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 0x7ff2800c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff2873dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800cdb10 con 0x7ff2800cd6a0 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff2873dd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2800b3110 con 0x7ff28015cc20 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff2873dd700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2800ccb20 con 0x7ff2800cf450 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff285179700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff28015cc20 0x7ff2800cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff284978700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff2800cd6a0 0x7ff2800ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49564/0 (socket says 172.21.15.16:49564) 2023-10-06T11:42:38.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.839+0000 7ff284978700 1 -- 172.21.15.16:0/1804410651 learned_addr learned my addr 172.21.15.16:0/1804410651 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff28597a700 1 --2- 172.21.15.16:0/1804410651 >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 0x7ff2800c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff285179700 1 -- 172.21.15.16:0/1804410651 >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 msgr2=0x7ff2800c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff285179700 1 --2- 172.21.15.16:0/1804410651 >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 0x7ff2800c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff285179700 1 -- 172.21.15.16:0/1804410651 >> v1:172.21.15.16:6789/0 conn(0x7ff2800cd6a0 legacy=0x7ff2800ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff27ffff700 1 -- 172.21.15.16:0/1804410651 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285959887 0 0) 0x7ff2800cdb10 con 0x7ff2800cd6a0 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff285179700 1 -- 172.21.15.16:0/1804410651 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800cf880 con 0x7ff28015cc20 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff28597a700 1 --2- 172.21.15.16:0/1804410651 >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 0x7ff2800c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff285179700 1 --2- 172.21.15.16:0/1804410651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800cd160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff28009a240 tx=0x7ff2780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29b66db2691974b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff27ffff700 1 -- 172.21.15.16:0/1804410651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800b6b0 con 0x7ff28015cc20 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff27ffff700 1 -- 172.21.15.16:0/1804410651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800b810 con 0x7ff28015cc20 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff27ffff700 1 -- 172.21.15.16:0/1804410651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800bbb0 con 0x7ff28015cc20 2023-10-06T11:42:38.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff2873dd700 1 -- 172.21.15.16:0/1804410651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 msgr2=0x7ff2800cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.840+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1804410651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800cd160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff28009a240 tx=0x7ff2780091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 -- 172.21.15.16:0/1804410651 shutdown_connections 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1804410651 >> v2:172.21.15.177:3300/0 conn(0x7ff2800cf450 0x7ff2800c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1804410651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800cd160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 -- 172.21.15.16:0/1804410651 >> 172.21.15.16:0/1804410651 conn(0x7ff280099b40 msgr2=0x7ff28006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 -- 172.21.15.16:0/1804410651 shutdown_connections 2023-10-06T11:42:38.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.841+0000 7ff2873dd700 1 -- 172.21.15.16:0/1804410651 wait complete. 2023-10-06T11:42:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.842+0000 7ff2873dd700 1 Processor -- start 2023-10-06T11:42:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.842+0000 7ff2873dd700 1 -- start start 2023-10-06T11:42:38.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2800cd6a0 0x7ff280158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 0x7ff28016cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280156dd0 con 0x7ff28015cc20 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280169670 con 0x7ff2800cf450 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800b59e0 con 0x7ff2800cd6a0 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60378/0 (socket says 172.21.15.16:60378) 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 -- 172.21.15.16:0/1594571348 learned_addr learned my addr 172.21.15.16:0/1594571348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff284978700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 0x7ff28016cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 -- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2800cd6a0 msgr2=0x7ff280158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2800cd6a0 0x7ff280158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 -- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 msgr2=0x7ff28016cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 0x7ff28016cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 -- 172.21.15.16:0/1594571348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff278009420 con 0x7ff28015cc20 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff284978700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 0x7ff28016cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff28597a700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff27000cf20 tx=0x7ff270007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000adf0 con 0x7ff28015cc20 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270016420 con 0x7ff28015cc20 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28016d370 con 0x7ff28015cc20 2023-10-06T11:42:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2700167e0 con 0x7ff28015cc20 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.843+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28016d930 con 0x7ff28015cc20 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.844+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff28016db50 con 0x7ff28015cc20 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.845+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff270016bd0 con 0x7ff28015cc20 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.845+0000 7ff27d7fa700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.845+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff270012070 con 0x7ff28015cc20 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.845+0000 7ff285179700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:38.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.846+0000 7ff285179700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 0x7ff26c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2800c6410 tx=0x7ff278017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:38.849+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff27004e5a0 con 0x7ff28015cc20 2023-10-06T11:42:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.121+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff28004c380 con 0x7ff28015cc20 2023-10-06T11:42:39.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.121+0000 7ff27d7fa700 1 -- 172.21.15.16:0/1594571348 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff270052140 con 0x7ff28015cc20 2023-10-06T11:42:39.123 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9ltSYCCBAA0LbHS7AfdXp0X5kyPd1QVQ==", 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.124 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9loZRoExAAcVM5vfC8B3rscVnEOsVkqg==", 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9lxKLhDhAA/nrhDdOuamujt6Bn65YDHg==", 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.126 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9lwkwrGhAAp+yW5zQWHG9RflVm/EKYyA==", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBP8h9lNwF6ABAAQSocwfWfDqRCUFZNXVYMUA==", 2023-10-06T11:42:39.127 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBR8h9lylNcOhAA6MVxZoYsfNNH4Ba1jtGcTg==", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.128 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBU8h9lpWeCOBAAZ8qBWvfpx/Z0MrN1m4jMIA==", 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.129 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBX8h9lyZVuNhAALXqD6pr+4wTEqa5Yps7K7Q==", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBd8h9l3dfRNhAASUc1GOR8xGzcsAV55z8YAQ==", 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.130 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBg8h9l8UaLNRAAq/hqizhAVtfkujBKnTznDA==", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBj8h9lb3kCNhAAI6n2UoNJ9lbq+GnReN1neA==", 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-10-06T11:42:39.132 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBm8h9liT3eNRAASOGudkgH0uyd405p/NIqaQ==", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9lvHkHHhAAjOHhMp7Y737Y8jx3RQR1uQ==", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.133 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-10-06T11:42:39.134 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-10-06T11:42:39.134 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-10-06T11:42:39.136 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.136 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.136 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-10-06T11:42:39.136 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9lqCCMJhAATgZVBarXJ3fS0xLhA7UidA==", 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-10-06T11:42:39.137 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9llgR7IxAA6k0an30dLNvVRta4W6S+QA==", 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.138 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.139 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9lS972KxAA8CoEqWxe/erJBuYO+g3lcQ==", 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBM8h9lq6GIKBAAir4emvhBNrWtxWRGxTRdbw==", 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-10-06T11:42:39.141 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9lbDWsJhAABkjmzaYr7JFu8siuf+XoJA==", 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.142 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9lsj+sJhAAMpMcDRJ8tn8M8Pyi6oTtGQ==", 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.143 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9ld0usJhAAGDsIDBDAuNKQhaHN9G9QGg==", 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9luFSsJhAAVYcJMGedWQzCxHmifL5pSQ==", 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.145 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9lAF6sJhAAU82/7jsYZ9OBhR3RxuiUAw==", 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBx8h9lRmesJhAAM46Egym9EkNPxQT15Xd3vw==", 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T11:42:39.146 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9l8rTmBBAA3basAsynbmSDIeHx2VWeaA==", 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:39.147 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQBN8h9l02ZvARAAjJeuClekdfh0vdVcSsCD4Q==", 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:42:39.148 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.123+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 msgr2=0x7ff26c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.123+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 0x7ff26c0693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff2800c6410 tx=0x7ff278017000 comp rx=0 tx=0).stop 2023-10-06T11:42:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.123+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 msgr2=0x7ff2800b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.123+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff27000cf20 tx=0x7ff270007860 comp rx=0 tx=0).stop 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 shutdown_connections 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff28015cc20 0x7ff2800b54a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff26c066f30 0x7ff26c0693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2800cd6a0 0x7ff280158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 --2- 172.21.15.16:0/1594571348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2800cf450 0x7ff28016cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 >> 172.21.15.16:0/1594571348 conn(0x7ff280099b40 msgr2=0x7ff2800c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 shutdown_connections 2023-10-06T11:42:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.124+0000 7ff2873dd700 1 -- 172.21.15.16:0/1594571348 wait complete. 2023-10-06T11:42:39.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T11:42:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.270+0000 7f69f4290700 1 Processor -- start 2023-10-06T11:42:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.270+0000 7f69f4290700 1 -- start start 2023-10-06T11:42:39.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.270+0000 7f69f4290700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 0x7f69ec0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f4290700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ec0cd8a0 0x7f69ec06d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f4290700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69ec0cf060 con 0x7f69ec06cbf0 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f4290700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec0cdcc0 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f4290700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69ec154580 con 0x7f69ec15ab50 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ec0cd8a0 0x7f69ec06d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f69ec0cd8a0 0x7f69ec06d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60380/0 (socket says 172.21.15.16:60380) 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 -- 172.21.15.16:0/2509686909 learned_addr learned my addr 172.21.15.16:0/2509686909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f202c700 1 --2- 172.21.15.16:0/2509686909 >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 0x7f69ec0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 -- 172.21.15.16:0/2509686909 >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 msgr2=0x7f69ec0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 --2- 172.21.15.16:0/2509686909 >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 0x7f69ec0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 -- 172.21.15.16:0/2509686909 >> v1:172.21.15.177:6789/0 conn(0x7f69ec06cbf0 legacy=0x7f69ec0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 -- 172.21.15.16:0/2509686909 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec0cf900 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f202c700 1 --2- 172.21.15.16:0/2509686909 >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 0x7f69ec0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:39.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f282d700 1 --2- 172.21.15.16:0/2509686909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec06d040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69ec089740 tx=0x7f69e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bcfeaebcf63b26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f102a700 1 -- 172.21.15.16:0/2509686909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e070 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f102a700 1 -- 172.21.15.16:0/2509686909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e400ba40 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.271+0000 7f69f102a700 1 -- 172.21.15.16:0/2509686909 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40092e0 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 -- 172.21.15.16:0/2509686909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec0cd8a0 msgr2=0x7f69ec06d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 --2- 172.21.15.16:0/2509686909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec06d040 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69ec089740 tx=0x7f69e4009840 comp rx=0 tx=0).stop 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 -- 172.21.15.16:0/2509686909 shutdown_connections 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 --2- 172.21.15.16:0/2509686909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec06d040 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 --2- 172.21.15.16:0/2509686909 >> v2:172.21.15.177:3300/0 conn(0x7f69ec15ab50 0x7f69ec0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 -- 172.21.15.16:0/2509686909 >> 172.21.15.16:0/2509686909 conn(0x7f69ec08a270 msgr2=0x7f69ec099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 -- 172.21.15.16:0/2509686909 shutdown_connections 2023-10-06T11:42:39.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.272+0000 7f69f4290700 1 -- 172.21.15.16:0/2509686909 wait complete. 2023-10-06T11:42:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.273+0000 7f69f4290700 1 Processor -- start 2023-10-06T11:42:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.273+0000 7f69f4290700 1 -- start start 2023-10-06T11:42:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69ec15ab50 0x7f69ec0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c5100 con 0x7f69ec06cbf0 2023-10-06T11:42:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0c77f0 con 0x7f69ec0cd8a0 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f4290700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec0ca6d0 con 0x7f69ec15ab50 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60392/0 (socket says 172.21.15.16:60392) 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 -- 172.21.15.16:0/2468580138 learned_addr learned my addr 172.21.15.16:0/2468580138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f282d700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69ec15ab50 0x7f69ec0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f182b700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 -- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69ec15ab50 msgr2=0x7f69ec0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69ec15ab50 0x7f69ec0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 -- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69ec0cd8a0 msgr2=0x7f69ec0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec0c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 -- 172.21.15.16:0/2468580138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4009ae0 con 0x7f69ec06cbf0 2023-10-06T11:42:39.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69f202c700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69e800d890 tx=0x7f69e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.274+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8005a00 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.275+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8005b60 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.275+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec0c5350 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.275+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80125f0 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.275+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec154070 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.276+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec154290 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.276+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69e80129a0 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.276+0000 7f69e27fc700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 0x7f69d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.276+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f69e8084150 con 0x7f69ec06cbf0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.277+0000 7f69f182b700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 0x7f69d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.277+0000 7f69f182b700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 0x7f69d00693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0ca120 tx=0x7f69dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.281+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e804d5a0 con 0x7f69ec06cbf0 2023-10-06T11:42:39.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.554+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f69ec063e70 con 0x7f69ec06cbf0 2023-10-06T11:42:39.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.557+0000 7f69e27fc700 1 -- 172.21.15.16:0/2468580138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f69e8051510 con 0x7f69ec06cbf0 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 msgr2=0x7f69d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 0x7f69d00693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0ca120 tx=0x7f69dc009040 comp rx=0 tx=0).stop 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 msgr2=0x7f69ec153100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f69e800d890 tx=0x7f69e8008040 comp rx=0 tx=0).stop 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 shutdown_connections 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f69ec06cbf0 0x7f69ec153100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f69d0066f30 0x7f69d00693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f69ec15ab50 0x7f69ec0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 --2- 172.21.15.16:0/2468580138 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69ec0cd8a0 0x7f69ec0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 >> 172.21.15.16:0/2468580138 conn(0x7f69ec08a270 msgr2=0x7f69ec0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 shutdown_connections 2023-10-06T11:42:39.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.559+0000 7f69f4290700 1 -- 172.21.15.16:0/2468580138 wait complete. 2023-10-06T11:42:39.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T11:42:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 Processor -- start 2023-10-06T11:42:39.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 -- start start 2023-10-06T11:42:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f3006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f300ce1e0 0x7f6f300cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f300c88e0 con 0x7f6f30159240 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f300c8d70 con 0x7f6f300ce1e0 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.692+0000 7f6f3520c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300c7cd0 con 0x7f6f3006c5d0 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f300ce1e0 0x7f6f300cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f300ce1e0 0x7f6f300cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60408/0 (socket says 172.21.15.16:60408) 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 -- 172.21.15.16:0/235205305 learned_addr learned my addr 172.21.15.16:0/235205305 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2ed9d700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f3006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 -- 172.21.15.16:0/235205305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 msgr2=0x7f6f3006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f3006c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 -- 172.21.15.16:0/235205305 >> v1:172.21.15.177:6789/0 conn(0x7f6f30159240 legacy=0x7f6f30159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 -- 172.21.15.16:0/235205305 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f300c7fb0 con 0x7f6f300ce1e0 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2ed9d700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f3006c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.693+0000 7f6f2e59c700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f300ce1e0 0x7f6f300cb790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099b0 tx=0x7f6f24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718171548c6abc99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f2dd9b700 1 -- 172.21.15.16:0/235205305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400b860 con 0x7f6f300ce1e0 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f2dd9b700 1 -- 172.21.15.16:0/235205305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2400b9c0 con 0x7f6f300ce1e0 2023-10-06T11:42:39.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f2dd9b700 1 -- 172.21.15.16:0/235205305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2400bd60 con 0x7f6f300ce1e0 2023-10-06T11:42:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 -- 172.21.15.16:0/235205305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f300ce1e0 msgr2=0x7f6f300cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f300ce1e0 0x7f6f300cb790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6f240099b0 tx=0x7f6f24009250 comp rx=0 tx=0).stop 2023-10-06T11:42:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 -- 172.21.15.16:0/235205305 shutdown_connections 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f3006c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 --2- 172.21.15.16:0/235205305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f300ce1e0 0x7f6f300cb790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.694+0000 7f6f3520c700 1 -- 172.21.15.16:0/235205305 >> 172.21.15.16:0/235205305 conn(0x7f6f3006d520 msgr2=0x7f6f3006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.695+0000 7f6f3520c700 1 -- 172.21.15.16:0/235205305 shutdown_connections 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.695+0000 7f6f3520c700 1 -- 172.21.15.16:0/235205305 wait complete. 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.696+0000 7f6f3520c700 1 Processor -- start 2023-10-06T11:42:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.696+0000 7f6f3520c700 1 -- start start 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f30154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f300ce1e0 0x7f6f3016df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f300b31e0 con 0x7f6f30159240 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f30151b50 con 0x7f6f300ce1e0 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f3520c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3016e4a0 con 0x7f6f3006c5d0 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2e59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f300ce1e0 0x7f6f3016df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60412/0 (socket says 172.21.15.16:60412) 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2ed9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f30154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 -- 172.21.15.16:0/1525685858 learned_addr learned my addr 172.21.15.16:0/1525685858 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 -- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 msgr2=0x7f6f30154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f30154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 -- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f300ce1e0 msgr2=0x7f6f3016df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f300ce1e0 0x7f6f3016df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.697+0000 7f6f2f59e700 1 -- 172.21.15.16:0/1525685858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240094f0 con 0x7f6f30159240 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f2f59e700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c870 tx=0x7f6f28001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28012630 con 0x7f6f30159240 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f280127b0 con 0x7f6f30159240 2023-10-06T11:42:39.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28012b70 con 0x7f6f30159240 2023-10-06T11:42:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f300ade00 con 0x7f6f30159240 2023-10-06T11:42:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.698+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f300ad4d0 con 0x7f6f30159240 2023-10-06T11:42:39.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.699+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f300c77e0 con 0x7f6f30159240 2023-10-06T11:42:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.699+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f28003030 con 0x7f6f30159240 2023-10-06T11:42:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.700+0000 7f6f16ffd700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 0x7f6f1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.700+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6f28084210 con 0x7f6f30159240 2023-10-06T11:42:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.700+0000 7f6f2ed9d700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 0x7f6f1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:39.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.701+0000 7f6f2ed9d700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 0x7f6f1c0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f20006ea0 tx=0x7f6f20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:39.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.704+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f28051670 con 0x7f6f30159240 2023-10-06T11:42:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.980+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f6f300adf90 con 0x7f6f30159240 2023-10-06T11:42:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.983+0000 7f6f16ffd700 1 -- 172.21.15.16:0/1525685858 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f28054c90 con 0x7f6f30159240 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 msgr2=0x7f6f1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 0x7f6f1c0693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f20006ea0 tx=0x7f6f20008040 comp rx=0 tx=0).stop 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 msgr2=0x7f6f301536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800c870 tx=0x7f6f28001670 comp rx=0 tx=0).stop 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 shutdown_connections 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f30159240 0x7f6f301536a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6f1c066f30 0x7f6f1c0693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6f3006c5d0 0x7f6f30154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 --2- 172.21.15.16:0/1525685858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f300ce1e0 0x7f6f3016df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 >> 172.21.15.16:0/1525685858 conn(0x7f6f3006d520 msgr2=0x7f6f3006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 shutdown_connections 2023-10-06T11:42:39.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:39.985+0000 7f6f3520c700 1 -- 172.21.15.16:0/1525685858 wait complete. 2023-10-06T11:42:39.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T11:42:40.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.118+0000 7ff249193700 1 Processor -- start 2023-10-06T11:42:40.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.118+0000 7ff249193700 1 -- start start 2023-10-06T11:42:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff249193700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff24406d300 0x7ff2440cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff249193700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2440cd190 0x7ff2440cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff249193700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440cf060 con 0x7ff24415ab50 2023-10-06T11:42:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff249193700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff244154580 con 0x7ff2440cd190 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff249193700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440cf900 con 0x7ff24406d300 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff242d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff24415ab50 0x7ff24406cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49568/0 (socket says 172.21.15.16:49568) 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff242d9d700 1 -- 172.21.15.16:0/4131427440 learned_addr learned my addr 172.21.15.16:0/4131427440 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 --2- 172.21.15.16:0/4131427440 >> v2:172.21.15.16:3300/0 conn(0x7ff2440cd190 0x7ff2440cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24259c700 1 --2- 172.21.15.16:0/4131427440 >> v2:172.21.15.177:3300/0 conn(0x7ff24406d300 0x7ff2440cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff241d9b700 1 -- 172.21.15.16:0/4131427440 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264020923 0 0) 0x7ff2440cf060 con 0x7ff24415ab50 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff241d9b700 1 -- 172.21.15.16:0/4131427440 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff22c003570 con 0x7ff24415ab50 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 -- 172.21.15.16:0/4131427440 >> v2:172.21.15.177:3300/0 conn(0x7ff24406d300 msgr2=0x7ff2440cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 --2- 172.21.15.16:0/4131427440 >> v2:172.21.15.177:3300/0 conn(0x7ff24406d300 0x7ff2440cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 -- 172.21.15.16:0/4131427440 >> v1:172.21.15.16:6789/0 conn(0x7ff24415ab50 legacy=0x7ff24406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 -- 172.21.15.16:0/4131427440 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440cdec0 con 0x7ff2440cd190 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff24359e700 1 --2- 172.21.15.16:0/4131427440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440cd5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff24409e550 tx=0x7ff23c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38140abd774ad019 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff241d9b700 1 -- 172.21.15.16:0/4131427440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c00e070 con 0x7ff2440cd190 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff241d9b700 1 -- 172.21.15.16:0/4131427440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00bb60 con 0x7ff2440cd190 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.119+0000 7ff241d9b700 1 -- 172.21.15.16:0/4131427440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c012430 con 0x7ff2440cd190 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 -- 172.21.15.16:0/4131427440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 msgr2=0x7ff2440cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 --2- 172.21.15.16:0/4131427440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440cd5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff24409e550 tx=0x7ff23c009780 comp rx=0 tx=0).stop 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 -- 172.21.15.16:0/4131427440 shutdown_connections 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 --2- 172.21.15.16:0/4131427440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440cd5d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 --2- 172.21.15.16:0/4131427440 >> v2:172.21.15.177:3300/0 conn(0x7ff24406d300 0x7ff2440cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 -- 172.21.15.16:0/4131427440 >> 172.21.15.16:0/4131427440 conn(0x7ff24408a270 msgr2=0x7ff244099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 -- 172.21.15.16:0/4131427440 shutdown_connections 2023-10-06T11:42:40.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.120+0000 7ff249193700 1 -- 172.21.15.16:0/4131427440 wait complete. 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.121+0000 7ff249193700 1 Processor -- start 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.121+0000 7ff249193700 1 -- start start 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 0x7ff2440b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff24415ab50 0x7ff2440c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b4930 con 0x7ff2440cd190 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b3f60 con 0x7ff24406d300 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff249193700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440b3630 con 0x7ff24415ab50 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60438/0 (socket says 172.21.15.16:60438) 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 -- 172.21.15.16:0/3944747399 learned_addr learned my addr 172.21.15.16:0/3944747399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff242d9d700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 0x7ff2440b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24359e700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff24415ab50 0x7ff2440c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 -- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff24415ab50 msgr2=0x7ff2440c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff24415ab50 0x7ff2440c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 -- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 msgr2=0x7ff2440b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 0x7ff2440b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.122+0000 7ff24259c700 1 -- 172.21.15.16:0/3944747399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009a20 con 0x7ff2440cd190 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff242d9d700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 0x7ff2440b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff24259c700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff23800d860 tx=0x7ff238008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238009610 con 0x7ff2440cd190 2023-10-06T11:42:40.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff238009770 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2440b3910 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238009b30 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.123+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440ca530 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.124+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2440b40f0 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.124+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff23801e460 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.124+0000 7ff22affd700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 0x7ff2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.125+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff238084970 con 0x7ff2440cd190 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.125+0000 7ff242d9d700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 0x7ff2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.125+0000 7ff242d9d700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 0x7ff2300693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff234001360 tx=0x7ff234009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.129+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff238051dd0 con 0x7ff2440cd190 2023-10-06T11:42:40.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.403+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff244063e70 con 0x7ff2440cd190 2023-10-06T11:42:40.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.407+0000 7ff22affd700 1 -- 172.21.15.16:0/3944747399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff2380553f0 con 0x7ff2440cd190 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 msgr2=0x7ff2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 0x7ff2300693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff234001360 tx=0x7ff234009040 comp rx=0 tx=0).stop 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 msgr2=0x7ff2440b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff23800d860 tx=0x7ff238008040 comp rx=0 tx=0).stop 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 shutdown_connections 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff230066f30 0x7ff2300693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff24415ab50 0x7ff2440c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24406d300 0x7ff2440b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 --2- 172.21.15.16:0/3944747399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2440cd190 0x7ff2440b43f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 >> 172.21.15.16:0/3944747399 conn(0x7ff24408a270 msgr2=0x7ff2440d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 shutdown_connections 2023-10-06T11:42:40.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.409+0000 7ff249193700 1 -- 172.21.15.16:0/3944747399 wait complete. 2023-10-06T11:42:40.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T11:42:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.541+0000 7ff215de0700 1 Processor -- start 2023-10-06T11:42:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.541+0000 7ff215de0700 1 -- start start 2023-10-06T11:42:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.541+0000 7ff215de0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2100cd8a0 0x7ff2100cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff215de0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2100cf060 con 0x7ff21006d300 2023-10-06T11:42:40.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff215de0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2100cf900 con 0x7ff21015ab50 2023-10-06T11:42:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff215de0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2100cdce0 con 0x7ff2100cd8a0 2023-10-06T11:42:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2100cd8a0 0x7ff2100cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff21006d300 0x7ff2100cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49570/0 (socket says 172.21.15.16:49570) 2023-10-06T11:42:40.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20effd700 1 -- 172.21.15.16:0/3973300230 learned_addr learned my addr 172.21.15.16:0/3973300230 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20ffff700 1 -- 172.21.15.16:0/3973300230 >> v1:172.21.15.177:6789/0 conn(0x7ff21015ab50 legacy=0x7ff21006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20ffff700 1 -- 172.21.15.16:0/3973300230 >> v1:172.21.15.16:6789/0 conn(0x7ff21006d300 legacy=0x7ff2100cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20ffff700 1 -- 172.21.15.16:0/3973300230 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100cdec0 con 0x7ff2100cd8a0 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20ffff700 1 --2- 172.21.15.16:0/3973300230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2100cd8a0 0x7ff2100cfd90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff208009e60 tx=0x7ff208009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7228b36cd39661da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20e7fc700 1 -- 172.21.15.16:0/3973300230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800e070 con 0x7ff2100cd8a0 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20e7fc700 1 -- 172.21.15.16:0/3973300230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20800bc30 con 0x7ff2100cd8a0 2023-10-06T11:42:40.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.542+0000 7ff20e7fc700 1 -- 172.21.15.16:0/3973300230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208012430 con 0x7ff2100cd8a0 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 -- 172.21.15.16:0/3973300230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2100cd8a0 msgr2=0x7ff2100cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 --2- 172.21.15.16:0/3973300230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2100cd8a0 0x7ff2100cfd90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff208009e60 tx=0x7ff208009780 comp rx=0 tx=0).stop 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 -- 172.21.15.16:0/3973300230 shutdown_connections 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 --2- 172.21.15.16:0/3973300230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2100cd8a0 0x7ff2100cfd90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 -- 172.21.15.16:0/3973300230 >> 172.21.15.16:0/3973300230 conn(0x7ff21008a270 msgr2=0x7ff210099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 -- 172.21.15.16:0/3973300230 shutdown_connections 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.543+0000 7ff215de0700 1 -- 172.21.15.16:0/3973300230 wait complete. 2023-10-06T11:42:40.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.544+0000 7ff215de0700 1 Processor -- start 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.544+0000 7ff215de0700 1 -- start start 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21006d300 0x7ff210151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2100cd8a0 0x7ff210152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100b4a00 con 0x7ff21015ab50 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210151d70 con 0x7ff21006d300 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff215de0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101694b0 con 0x7ff2100cd8a0 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60452/0 (socket says 172.21.15.16:60452) 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 -- 172.21.15.16:0/2528688573 learned_addr learned my addr 172.21.15.16:0/2528688573 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:40.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20effd700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2100cd8a0 0x7ff210152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20f7fe700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21006d300 0x7ff210151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 -- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2100cd8a0 msgr2=0x7ff210152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2100cd8a0 0x7ff210152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 -- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21006d300 msgr2=0x7ff210151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21006d300 0x7ff210151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.545+0000 7ff20ffff700 1 -- 172.21.15.16:0/2528688573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208009a20 con 0x7ff21015ab50 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff20ffff700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2080097c0 tx=0x7ff208019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff208012e10 con 0x7ff21015ab50 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20801e630 con 0x7ff21015ab50 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20801e9f0 con 0x7ff21015ab50 2023-10-06T11:42:40.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2100b3df0 con 0x7ff21015ab50 2023-10-06T11:42:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.546+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2100b43b0 con 0x7ff21015ab50 2023-10-06T11:42:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.547+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2100b4b70 con 0x7ff21015ab50 2023-10-06T11:42:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.547+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff208024070 con 0x7ff21015ab50 2023-10-06T11:42:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.548+0000 7ff1f77fe700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 0x7ff1fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.548+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff208086b30 con 0x7ff21015ab50 2023-10-06T11:42:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.548+0000 7ff20f7fe700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 0x7ff1fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.548+0000 7ff20f7fe700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 0x7ff1fc0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fd0 tx=0x7ff200007eb0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.552+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff208053f90 con 0x7ff21015ab50 2023-10-06T11:42:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.822+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ff210063e70 con 0x7ff21015ab50 2023-10-06T11:42:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.826+0000 7ff1f77fe700 1 -- 172.21.15.16:0/2528688573 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff20802c0a0 con 0x7ff21015ab50 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 msgr2=0x7ff1fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 0x7ff1fc0693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff200007fd0 tx=0x7ff200007eb0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 msgr2=0x7ff2100b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2080097c0 tx=0x7ff208019000 comp rx=0 tx=0).stop 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 shutdown_connections 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21015ab50 0x7ff2100b5c70 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff1fc066f30 0x7ff1fc0693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff21006d300 0x7ff210151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 --2- 172.21.15.16:0/2528688573 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2100cd8a0 0x7ff210152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 >> 172.21.15.16:0/2528688573 conn(0x7ff21008a270 msgr2=0x7ff210153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 shutdown_connections 2023-10-06T11:42:40.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.828+0000 7ff215de0700 1 -- 172.21.15.16:0/2528688573 wait complete. 2023-10-06T11:42:40.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T11:42:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 Processor -- start 2023-10-06T11:42:40.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 -- start start 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680cc990 con 0x7f886815a2a0 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8868157e00 con 0x7f8868150960 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.960+0000 7f886d90c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886815a780 con 0x7f88680cbe20 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88677fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88667fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f886815a2a0 0x7f88680cb710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49586/0 (socket says 172.21.15.16:49586) 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88667fc700 1 -- 172.21.15.16:0/248482502 learned_addr learned my addr 172.21.15.16:0/248482502 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88677fe700 1 -- 172.21.15.16:0/248482502 >> v1:172.21.15.177:6789/0 conn(0x7f8868150960 legacy=0x7f88681576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88677fe700 1 -- 172.21.15.16:0/248482502 >> v1:172.21.15.16:6789/0 conn(0x7f886815a2a0 legacy=0x7f88680cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f8865ffb700 1 -- 172.21.15.16:0/248482502 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529354930 0 0) 0x7f88680cc990 con 0x7f886815a2a0 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88677fe700 1 -- 172.21.15.16:0/248482502 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886815aa60 con 0x7f88680cbe20 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f88677fe700 1 --2- 172.21.15.16:0/248482502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8858009910 tx=0x7f88580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df6896511d0b4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f8865ffb700 1 -- 172.21.15.16:0/248482502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800b7e0 con 0x7f88680cbe20 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f8865ffb700 1 -- 172.21.15.16:0/248482502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885800b940 con 0x7f88680cbe20 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.961+0000 7f8865ffb700 1 -- 172.21.15.16:0/248482502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885800bce0 con 0x7f88680cbe20 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 -- 172.21.15.16:0/248482502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cbe20 msgr2=0x7f8868150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 --2- 172.21.15.16:0/248482502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8858009910 tx=0x7f88580091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 -- 172.21.15.16:0/248482502 shutdown_connections 2023-10-06T11:42:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 --2- 172.21.15.16:0/248482502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 -- 172.21.15.16:0/248482502 >> 172.21.15.16:0/248482502 conn(0x7f8868068350 msgr2=0x7f8868068750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 -- 172.21.15.16:0/248482502 shutdown_connections 2023-10-06T11:42:40.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.962+0000 7f886d90c700 1 -- 172.21.15.16:0/248482502 wait complete. 2023-10-06T11:42:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.963+0000 7f886d90c700 1 Processor -- start 2023-10-06T11:42:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.963+0000 7f886d90c700 1 -- start start 2023-10-06T11:42:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8868150960 0x7f886813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88680c7440 con 0x7f886815a2a0 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868169e60 con 0x7f88680cbe20 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f886d90c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88681539d0 con 0x7f8868150960 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60472/0 (socket says 172.21.15.16:60472) 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 -- 172.21.15.16:0/966165902 learned_addr learned my addr 172.21.15.16:0/966165902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f8866ffd700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88667fc700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8868150960 0x7f886813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 -- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8868150960 msgr2=0x7f886813ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8868150960 0x7f886813ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 -- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88680cbe20 msgr2=0x7f8868150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.964+0000 7f88677fe700 1 -- 172.21.15.16:0/966165902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8858009420 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f88677fe700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88580093f0 tx=0x7f8858016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885801b450 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8858020070 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88680c6f50 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885801b820 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.965+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8868169810 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.966+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8868169ff0 con 0x7f886815a2a0 2023-10-06T11:42:40.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.966+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8858022030 con 0x7f886815a2a0 2023-10-06T11:42:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.967+0000 7f885effd700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 0x7f88480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.967+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f88580864d0 con 0x7f886815a2a0 2023-10-06T11:42:40.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.967+0000 7f8866ffd700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 0x7f88480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:40.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.967+0000 7f8866ffd700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 0x7f88480693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8860007fd0 tx=0x7f88600026a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:40.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:40.971+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8858053930 con 0x7f886815a2a0 2023-10-06T11:42:41.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.243+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8868169ff0 con 0x7f886815a2a0 2023-10-06T11:42:41.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.246+0000 7f885effd700 1 -- 172.21.15.16:0/966165902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8858056f50 con 0x7f886815a2a0 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 msgr2=0x7f88480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 0x7f88480693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8860007fd0 tx=0x7f88600026a0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 msgr2=0x7f8868155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f88580093f0 tx=0x7f8858016000 comp rx=0 tx=0).stop 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 shutdown_connections 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.248+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f886815a2a0 0x7f8868155bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8848066f30 0x7f88480693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88680cbe20 0x7f8868150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 --2- 172.21.15.16:0/966165902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8868150960 0x7f886813ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 >> 172.21.15.16:0/966165902 conn(0x7f8868068350 msgr2=0x7f8868068720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 shutdown_connections 2023-10-06T11:42:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.249+0000 7f886d90c700 1 -- 172.21.15.16:0/966165902 wait complete. 2023-10-06T11:42:41.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T11:42:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.381+0000 7f8c184cc700 1 Processor -- start 2023-10-06T11:42:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.381+0000 7f8c184cc700 1 -- start start 2023-10-06T11:42:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c184cc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c100ca870 0x7f8c10153e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c184cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c1015e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c184cc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c101543a0 con 0x7f8c10153a90 2023-10-06T11:42:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c184cc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1015a530 con 0x7f8c100ca870 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c184cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1015d480 con 0x7f8c1015dbf0 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c100ca870 0x7f8c10153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c100ca870 0x7f8c10153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60482/0 (socket says 172.21.15.16:60482) 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 -- 172.21.15.16:0/2556007449 learned_addr learned my addr 172.21.15.16:0/2556007449 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c16a69700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c1015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 -- 172.21.15.16:0/2556007449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 msgr2=0x7f8c1015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c1015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 -- 172.21.15.16:0/2556007449 >> v1:172.21.15.177:6789/0 conn(0x7f8c10153a90 legacy=0x7f8c10159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 -- 172.21.15.16:0/2556007449 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1015d760 con 0x7f8c100ca870 2023-10-06T11:42:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c16a69700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c1015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.382+0000 7f8c15a67700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10153e60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099b0 tx=0x7f8c00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d61b6a97eb7e949 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c15266700 1 -- 172.21.15.16:0/2556007449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000b860 con 0x7f8c100ca870 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c15266700 1 -- 172.21.15.16:0/2556007449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000b9c0 con 0x7f8c100ca870 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c15266700 1 -- 172.21.15.16:0/2556007449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000bd60 con 0x7f8c100ca870 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 -- 172.21.15.16:0/2556007449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 msgr2=0x7f8c10153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10153e60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099b0 tx=0x7f8c00009250 comp rx=0 tx=0).stop 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 -- 172.21.15.16:0/2556007449 shutdown_connections 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c1015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 --2- 172.21.15.16:0/2556007449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10153e60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 -- 172.21.15.16:0/2556007449 >> 172.21.15.16:0/2556007449 conn(0x7f8c1006cee0 msgr2=0x7f8c10067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 -- 172.21.15.16:0/2556007449 shutdown_connections 2023-10-06T11:42:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.383+0000 7f8c184cc700 1 -- 172.21.15.16:0/2556007449 wait complete. 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.384+0000 7f8c184cc700 1 Processor -- start 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.384+0000 7f8c184cc700 1 -- start start 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c10153a90 0x7f8c1016de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c10151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1016bf40 con 0x7f8c100ca870 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1016b330 con 0x7f8c10153a90 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c184cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1016b610 con 0x7f8c1015dbf0 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60494/0 (socket says 172.21.15.16:60494) 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 -- 172.21.15.16:0/3699881147 learned_addr learned my addr 172.21.15.16:0/3699881147 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c15a67700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c10153a90 0x7f8c1016de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16a69700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c10151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 -- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 msgr2=0x7f8c10151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c10151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 -- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c10153a90 msgr2=0x7f8c1016de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.385+0000 7f8c16268700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c10153a90 0x7f8c1016de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c16268700 1 -- 172.21.15.16:0/3699881147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000094f0 con 0x7f8c100ca870 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c16268700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c590 tx=0x7f8c0c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c011590 con 0x7f8c100ca870 2023-10-06T11:42:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c011710 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1016b8f0 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c011ad0 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.386+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c10167c70 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.387+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c10063e70 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.387+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c0c01b3b0 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.388+0000 7f8c067fc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 0x7f8bf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.388+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8c0c0837b0 con 0x7f8c100ca870 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.388+0000 7f8c15a67700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 0x7f8bf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.388+0000 7f8c15a67700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 0x7f8bf40693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099b0 tx=0x7f8c00017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.392+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c0c050c10 con 0x7f8c100ca870 2023-10-06T11:42:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.666+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f8c10151620 con 0x7f8c100ca870 2023-10-06T11:42:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.669+0000 7f8c067fc700 1 -- 172.21.15.16:0/3699881147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8c0c017020 con 0x7f8c100ca870 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.671+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 msgr2=0x7f8bf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.671+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 0x7f8bf40693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c000099b0 tx=0x7f8c00017000 comp rx=0 tx=0).stop 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.671+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 msgr2=0x7f8c10143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.671+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c590 tx=0x7f8c0c00c8a0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 shutdown_connections 2023-10-06T11:42:41.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c100ca870 0x7f8c10143460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8bf4066f30 0x7f8bf40693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8c1015dbf0 0x7f8c10151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 --2- 172.21.15.16:0/3699881147 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c10153a90 0x7f8c1016de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 >> 172.21.15.16:0/3699881147 conn(0x7f8c1006cee0 msgr2=0x7f8c10069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 shutdown_connections 2023-10-06T11:42:41.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.672+0000 7f8c184cc700 1 -- 172.21.15.16:0/3699881147 wait complete. 2023-10-06T11:42:41.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:41.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.803+0000 7f16d0f67700 1 Processor -- start 2023-10-06T11:42:41.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.803+0000 7f16d0f67700 1 -- start start 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16d0f67700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16cc15cc20 0x7f16cc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16d0f67700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0cdb10 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16d0f67700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16cc0ccb20 con 0x7f16cc0cf450 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16d0f67700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0b3110 con 0x7f16cc15cc20 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16ca59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16cc15cc20 0x7f16cc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16c9d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f16cc0cd6a0 0x7f16cc0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49588/0 (socket says 172.21.15.16:49588) 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.804+0000 7f16c9d9b700 1 -- 172.21.15.16:0/2369663363 learned_addr learned my addr 172.21.15.16:0/2369663363 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16ca59c700 1 -- 172.21.15.16:0/2369663363 >> v1:172.21.15.177:6789/0 conn(0x7f16cc0cf450 legacy=0x7f16cc0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16ca59c700 1 -- 172.21.15.16:0/2369663363 >> v1:172.21.15.16:6789/0 conn(0x7f16cc0cd6a0 legacy=0x7f16cc0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16ca59c700 1 -- 172.21.15.16:0/2369663363 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0cf880 con 0x7f16cc15cc20 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16c959a700 1 -- 172.21.15.16:0/2369663363 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513612285 0 0) 0x7f16cc0cdb10 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16ca59c700 1 --2- 172.21.15.16:0/2369663363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc15cc20 0x7f16cc0cd160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16c40098e0 tx=0x7f16c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea87146eeaa7166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16c959a700 1 -- 172.21.15.16:0/2369663363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4009e80 con 0x7f16cc15cc20 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16c959a700 1 -- 172.21.15.16:0/2369663363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c400b860 con 0x7f16cc15cc20 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16c959a700 1 -- 172.21.15.16:0/2369663363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c400bbe0 con 0x7f16cc15cc20 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16d0f67700 1 -- 172.21.15.16:0/2369663363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc15cc20 msgr2=0x7f16cc0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.805+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2369663363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc15cc20 0x7f16cc0cd160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16c40098e0 tx=0x7f16c40091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.806+0000 7f16d0f67700 1 -- 172.21.15.16:0/2369663363 shutdown_connections 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.806+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2369663363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc15cc20 0x7f16cc0cd160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.806+0000 7f16d0f67700 1 -- 172.21.15.16:0/2369663363 >> 172.21.15.16:0/2369663363 conn(0x7f16cc099b40 msgr2=0x7f16cc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.806+0000 7f16d0f67700 1 -- 172.21.15.16:0/2369663363 shutdown_connections 2023-10-06T11:42:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.806+0000 7f16d0f67700 1 -- 172.21.15.16:0/2369663363 wait complete. 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 Processor -- start 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 -- start start 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 0x7f16cc0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc15cc20 0x7f16cc169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc09e160 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c8700 con 0x7f16cc15cc20 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.807+0000 7f16d0f67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c7810 con 0x7f16cc0cf450 2023-10-06T11:42:41.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60504/0 (socket says 172.21.15.16:60504) 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 -- 172.21.15.16:0/2823740149 learned_addr learned my addr 172.21.15.16:0/2823740149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16c9d9b700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 0x7f16cc0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16cad9d700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc15cc20 0x7f16cc169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 -- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 msgr2=0x7f16cc0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 0x7f16cc0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 -- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc15cc20 msgr2=0x7f16cc169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc15cc20 0x7f16cc169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 -- 172.21.15.16:0/2823740149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4009420 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16c9d9b700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 0x7f16cc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16ca59c700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f16cc089390 tx=0x7f16c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c401f070 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c4000bc0 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4025420 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0c7af0 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.808+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0c80b0 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.809+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc04c380 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.810+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16c4023070 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.810+0000 7f16c27fc700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.810+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f16c40857c0 con 0x7f16cc0cd6a0 2023-10-06T11:42:41.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.810+0000 7f16c9d9b700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.811+0000 7f16c9d9b700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 0x7f16ac0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16b4001890 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:41.814+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16c4052c20 con 0x7f16cc0cd6a0 2023-10-06T11:42:42.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.011+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16cc0cbf10 con 0x7f16cc0cd6a0 2023-10-06T11:42:42.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.012+0000 7f16c27fc700 1 -- 172.21.15.16:0/2823740149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+13650 (secure 0 0 0) 0x7f16c4015090 con 0x7f16cc0cd6a0 2023-10-06T11:42:42.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:42.014 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:38.709427+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 msgr2=0x7f16ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 0x7f16ac0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f16b4001890 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 msgr2=0x7f16cc157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f16cc089390 tx=0x7f16c4017000 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 shutdown_connections 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16cc0cd6a0 0x7f16cc157c60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f16ac066f30 0x7f16ac0693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16cc15cc20 0x7f16cc169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 --2- 172.21.15.16:0/2823740149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f16cc0cf450 0x7f16cc0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 >> 172.21.15.16:0/2823740149 conn(0x7f16cc099b40 msgr2=0x7f16cc0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 shutdown_connections 2023-10-06T11:42:42.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.014+0000 7f16d0f67700 1 -- 172.21.15.16:0/2823740149 wait complete. 2023-10-06T11:42:42.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T11:42:42.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:42:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.149+0000 7f37c942a700 1 Processor -- start 2023-10-06T11:42:42.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.149+0000 7f37c942a700 1 -- start start 2023-10-06T11:42:42.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37c4159740 0x7f37c4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f37c40b4080 0x7f37c40b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c40b4a60 con 0x7f37c4159740 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c40b1790 con 0x7f37c40b4080 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c942a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c40b0e60 con 0x7f37c40b09d0 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37c4159740 0x7f37c4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37c4159740 0x7f37c4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60512/0 (socket says 172.21.15.16:60512) 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 -- 172.21.15.16:0/1333197281 learned_addr learned my addr 172.21.15.16:0/1333197281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c27fc700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c37fe700 1 --2- 172.21.15.16:0/1333197281 >> v2:172.21.15.177:3300/0 conn(0x7f37c40b4080 0x7f37c40b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 -- 172.21.15.16:0/1333197281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 msgr2=0x7f37c40b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c40b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 -- 172.21.15.16:0/1333197281 >> v2:172.21.15.177:3300/0 conn(0x7f37c40b4080 msgr2=0x7f37c40b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1333197281 >> v2:172.21.15.177:3300/0 conn(0x7f37c40b4080 0x7f37c40b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 -- 172.21.15.16:0/1333197281 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40b1140 con 0x7f37c4159740 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4159b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f37c40d0550 tx=0x7f37b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474e7f8947f54857 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.150+0000 7f37c1ffb700 1 -- 172.21.15.16:0/1333197281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400b6d0 con 0x7f37c4159740 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c1ffb700 1 -- 172.21.15.16:0/1333197281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b400b830 con 0x7f37c4159740 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c1ffb700 1 -- 172.21.15.16:0/1333197281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400bbf0 con 0x7f37c4159740 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 -- 172.21.15.16:0/1333197281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 msgr2=0x7f37c4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4159b10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f37c40d0550 tx=0x7f37b4009250 comp rx=0 tx=0).stop 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 -- 172.21.15.16:0/1333197281 shutdown_connections 2023-10-06T11:42:42.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 --2- 172.21.15.16:0/1333197281 >> v2:172.21.15.177:3300/0 conn(0x7f37c40b4080 0x7f37c40b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4159b10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 --2- 172.21.15.16:0/1333197281 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c40b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 -- 172.21.15.16:0/1333197281 >> 172.21.15.16:0/1333197281 conn(0x7f37c4099a40 msgr2=0x7f37c406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 -- 172.21.15.16:0/1333197281 shutdown_connections 2023-10-06T11:42:42.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.151+0000 7f37c942a700 1 -- 172.21.15.16:0/1333197281 wait complete. 2023-10-06T11:42:42.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.152+0000 7f37c942a700 1 Processor -- start 2023-10-06T11:42:42.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.152+0000 7f37c942a700 1 -- start start 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c4152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c40b4080 0x7f37c4151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c41521f0 con 0x7f37c4159740 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4153620 con 0x7f37c40b4080 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c942a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c4153900 con 0x7f37c40b09d0 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60522/0 (socket says 172.21.15.16:60522) 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 -- 172.21.15.16:0/1129394556 learned_addr learned my addr 172.21.15.16:0/1129394556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c27fc700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c40b4080 0x7f37c4151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c4152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 -- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 msgr2=0x7f37c4152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c4152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 -- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c40b4080 msgr2=0x7f37c4151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c40b4080 0x7f37c4151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.153+0000 7f37c37fe700 1 -- 172.21.15.16:0/1129394556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40094f0 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37c37fe700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009e90 tx=0x7f37bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cc20 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00f040 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c4153be0 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc014480 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.154+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c41541a0 con 0x7f37c4159740 2023-10-06T11:42:42.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.155+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c41543c0 con 0x7f37c4159740 2023-10-06T11:42:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.155+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37bc014830 con 0x7f37c4159740 2023-10-06T11:42:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.156+0000 7f37aaffd700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.156+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f37bc085040 con 0x7f37c4159740 2023-10-06T11:42:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.156+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.156+0000 7f37c2ffd700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 0x7f37b00693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37c40d0550 tx=0x7f37b4017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.160+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37bc055f30 con 0x7f37c4159740 2023-10-06T11:42:42.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.352+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f37c404c380 con 0x7f37c4159740 2023-10-06T11:42:42.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.353+0000 7f37aaffd700 1 -- 172.21.15.16:0/1129394556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f37bc055ac0 con 0x7f37c4159740 2023-10-06T11:42:42.355 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 msgr2=0x7f37b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 0x7f37b00693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37c40d0550 tx=0x7f37b4017000 comp rx=0 tx=0).stop 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 msgr2=0x7f37c4155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009e90 tx=0x7f37bc00b040 comp rx=0 tx=0).stop 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 shutdown_connections 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37c4159740 0x7f37c4155130 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37b0066f30 0x7f37b00693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37c40b09d0 0x7f37c4152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 --2- 172.21.15.16:0/1129394556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37c40b4080 0x7f37c4151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 >> 172.21.15.16:0/1129394556 conn(0x7f37c4099a40 msgr2=0x7f37c40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 shutdown_connections 2023-10-06T11:42:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.355+0000 7f37c942a700 1 -- 172.21.15.16:0/1129394556 wait complete. 2023-10-06T11:42:42.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:42:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.492+0000 7f907b9a2700 1 Processor -- start 2023-10-06T11:42:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.492+0000 7f907b9a2700 1 -- start start 2023-10-06T11:42:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.492+0000 7f907b9a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907b9a2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90740cb660 0x7f907415da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907b9a2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907413c9f0 con 0x7f90740cb290 2023-10-06T11:42:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907b9a2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907415e270 con 0x7f90740cb660 2023-10-06T11:42:42.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907b9a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907406dff0 con 0x7f90740cc7d0 2023-10-06T11:42:42.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90740cb660 0x7f907415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9078f3d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907973e700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f90740cb290 0x7f90740cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:41152/0 (socket says 172.21.15.16:41152) 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f90740cb660 0x7f907415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:50722/0 (socket says 172.21.15.16:50722) 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f907973e700 1 -- 172.21.15.16:0/745470563 learned_addr learned my addr 172.21.15.16:0/745470563 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9078f3d700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:36490/0 (socket says 172.21.15.16:36490) 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 -- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 msgr2=0x7f907415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 -- 172.21.15.16:0/745470563 >> v1:172.21.15.177:6789/0 conn(0x7f90740cb290 legacy=0x7f90740cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.493+0000 7f9079f3f700 1 -- 172.21.15.16:0/745470563 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907415ae30 con 0x7f90740cb660 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f9079f3f700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb660 0x7f907415da40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90640098e0 tx=0x7f90640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80600bcf41d0a60f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f9073fff700 1 -- 172.21.15.16:0/745470563 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064011630 con 0x7f90740cb660 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f9073fff700 1 -- 172.21.15.16:0/745470563 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064011790 con 0x7f90740cb660 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f9073fff700 1 -- 172.21.15.16:0/745470563 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064011b30 con 0x7f90740cb660 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 -- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb660 msgr2=0x7f907415da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb660 0x7f907415da40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90640098e0 tx=0x7f90640091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 -- 172.21.15.16:0/745470563 shutdown_connections 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb660 0x7f907415da40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 --2- 172.21.15.16:0/745470563 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 -- 172.21.15.16:0/745470563 >> 172.21.15.16:0/745470563 conn(0x7f9074067c90 msgr2=0x7f9074068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 -- 172.21.15.16:0/745470563 shutdown_connections 2023-10-06T11:42:42.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.494+0000 7f907b9a2700 1 -- 172.21.15.16:0/745470563 wait complete. 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.495+0000 7f907b9a2700 1 Processor -- start 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 -- start start 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 0x7f907416a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907415efb0 con 0x7f90740cb660 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907415e3c0 con 0x7f90740cb290 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.496+0000 7f907b9a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907415e610 con 0x7f90740cc7d0 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60536/0 (socket says 172.21.15.16:60536) 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 -- 172.21.15.16:0/878097235 learned_addr learned my addr 172.21.15.16:0/878097235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f907973e700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 0x7f907416a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9079f3f700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 -- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 msgr2=0x7f907415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 -- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 msgr2=0x7f907416a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 0x7f907416a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 -- 172.21.15.16:0/878097235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9064009420 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f907973e700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 0x7f907416a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f9078f3d700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f906800e930 tx=0x7f90680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800eed0 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906800c850 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906800cbf0 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907415e8f0 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.497+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907416ace0 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.498+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907415f0f0 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.499+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9068013470 con 0x7f90740cb660 2023-10-06T11:42:42.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.499+0000 7f90717fa700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 0x7f90600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.499+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9068084570 con 0x7f90740cb660 2023-10-06T11:42:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.499+0000 7f907973e700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 0x7f90600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.500+0000 7f907973e700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 0x7f90600693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f906c005c50 tx=0x7f906c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.503+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f907415f0f0 con 0x7f90740cb660 2023-10-06T11:42:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.701+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f907415f0f0 con 0x7f90740cb660 2023-10-06T11:42:42.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.702+0000 7f90717fa700 1 -- 172.21.15.16:0/878097235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f907415f0f0 con 0x7f90740cb660 2023-10-06T11:42:42.704 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:42:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 msgr2=0x7f90600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 0x7f90600693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f906c005c50 tx=0x7f906c009040 comp rx=0 tx=0).stop 2023-10-06T11:42:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 msgr2=0x7f907415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f906800e930 tx=0x7f90680091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 shutdown_connections 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9060066f30 0x7f90600693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f90740cc7d0 0x7f907415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90740cb290 0x7f907416a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 --2- 172.21.15.16:0/878097235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90740cb660 0x7f907415f3d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 >> 172.21.15.16:0/878097235 conn(0x7f9074067c90 msgr2=0x7f907406edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 shutdown_connections 2023-10-06T11:42:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.705+0000 7f907b9a2700 1 -- 172.21.15.16:0/878097235 wait complete. 2023-10-06T11:42:42.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T11:42:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.835+0000 7fd901ab2700 1 Processor -- start 2023-10-06T11:42:42.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.835+0000 7fd901ab2700 1 -- start start 2023-10-06T11:42:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd901ab2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8fc0cd870 0x7fd8fc0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd901ab2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0ccb20 con 0x7fd8fc15cc20 2023-10-06T11:42:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd901ab2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0cbf10 con 0x7fd8fc0cf280 2023-10-06T11:42:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd901ab2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b3110 con 0x7fd8fc0cd870 2023-10-06T11:42:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd8fb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8fc15cc20 0x7fd8fc0cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/1006018157 learned_addr learned my addr 172.21.15.16:0/1006018157 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.836+0000 7fd8faffd700 1 --2- 172.21.15.16:0/1006018157 >> v2:172.21.15.177:3300/0 conn(0x7fd8fc0cd870 0x7fd8fc0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119793397 0 0) 0x7fd8fc0ccb20 con 0x7fd8fc15cc20 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003570 con 0x7fd8fc15cc20 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2286473940 0 0) 0x7fd8e4003570 con 0x7fd8fc15cc20 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 >> v2:172.21.15.177:3300/0 conn(0x7fd8fc0cd870 msgr2=0x7fd8fc0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 --2- 172.21.15.16:0/1006018157 >> v2:172.21.15.177:3300/0 conn(0x7fd8fc0cd870 0x7fd8fc0ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 >> v1:172.21.15.177:6789/0 conn(0x7fd8fc0cf280 legacy=0x7fd8fc0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0cc1f0 con 0x7fd8fc15cc20 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fd8ec001470 con 0x7fd8fc15cc20 2023-10-06T11:42:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 >> v1:172.21.15.16:6789/0 conn(0x7fd8fc15cc20 legacy=0x7fd8fc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8e4004810 0x7fd8e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8e40052e0 0x7fd8e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0b3110 con 0x7fd8e4003a60 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400c2a0 con 0x7fd8e40052e0 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400c460 con 0x7fd8e4004810 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8faffd700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8e4004810 0x7fd8e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fbfff700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8e40052e0 0x7fd8e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8e4004810 msgr2=0x7fd8e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8e4004810 0x7fd8e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8e40052e0 msgr2=0x7fd8e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8e40052e0 0x7fd8e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fbfff700 1 -- 172.21.15.16:0/1006018157 reap_dead start 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.837+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/1006018157 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4003570 con 0x7fd8e4003a60 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8e4003e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c3c0 tx=0x7fd8ec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee2c13d166ec606 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00cc90 con 0x7fd8e4003a60 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00cdf0 con 0x7fd8e4003a60 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd8fa7fc700 1 -- 172.21.15.16:0/1006018157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec005b50 con 0x7fd8e4003a60 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 -- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 msgr2=0x7fd8e4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8e4003e70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c3c0 tx=0x7fd8ec001e50 comp rx=0 tx=0).stop 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 -- 172.21.15.16:0/1006018157 shutdown_connections 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 --2- 172.21.15.16:0/1006018157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8e4003e70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 -- 172.21.15.16:0/1006018157 >> 172.21.15.16:0/1006018157 conn(0x7fd8fc099b40 msgr2=0x7fd8fc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 -- 172.21.15.16:0/1006018157 shutdown_connections 2023-10-06T11:42:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.838+0000 7fd901ab2700 1 -- 172.21.15.16:0/1006018157 wait complete. 2023-10-06T11:42:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 Processor -- start 2023-10-06T11:42:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 -- start start 2023-10-06T11:42:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8fc06e240 0x7fd8fc09ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8fc06e6b0 0x7fd8fc09b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc09d800 con 0x7fd8e4003a60 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c8710 con 0x7fd8fc06e240 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.840+0000 7fd901ab2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc09b7f0 con 0x7fd8fc06e6b0 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60558/0 (socket says 172.21.15.16:60558) 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/4217012689 learned_addr learned my addr 172.21.15.16:0/4217012689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8faffd700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8fc06e240 0x7fd8fc09ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fbfff700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8fc06e6b0 0x7fd8fc09b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8fc06e6b0 msgr2=0x7fd8fc09b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8fc06e6b0 0x7fd8fc09b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8fc06e240 msgr2=0x7fd8fc09ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8fc06e240 0x7fd8fc09ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 -- 172.21.15.16:0/4217012689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc09d1b0 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8fb7fe700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c270 tx=0x7fd8ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec000bc0 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec0070d0 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007490 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc09d490 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.841+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc0ca190 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.842+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc09dae0 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.843+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8ec028460 con 0x7fd8e4003a60 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.843+0000 7fd8e37fe700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.843+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd8ec085810 con 0x7fd8e4003a60 2023-10-06T11:42:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.843+0000 7fd8faffd700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.844+0000 7fd8faffd700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc15dab0 tx=0x7fd8f00091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:42.848+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8ec052c70 con 0x7fd8e4003a60 2023-10-06T11:42:43.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.041+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd8fc04c380 con 0x7fd8e4003a60 2023-10-06T11:42:43.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.041+0000 7fd8e37fe700 1 -- 172.21.15.16:0/4217012689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd8ec017070 con 0x7fd8e4003a60 2023-10-06T11:42:43.043 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:43.043 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:43.044 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-10-06T11:42:43.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 msgr2=0x7fd8e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc15dab0 tx=0x7fd8f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 msgr2=0x7fd8fc09dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400c270 tx=0x7fd8ec015000 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 shutdown_connections 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8e4003a60 0x7fd8fc09dc90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8e8066f30 0x7fd8e80693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8fc06e6b0 0x7fd8fc09b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 --2- 172.21.15.16:0/4217012689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8fc06e240 0x7fd8fc09ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 >> 172.21.15.16:0/4217012689 conn(0x7fd8fc099b40 msgr2=0x7fd8fc0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 shutdown_connections 2023-10-06T11:42:43.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.043+0000 7fd901ab2700 1 -- 172.21.15.16:0/4217012689 wait complete. 2023-10-06T11:42:43.057 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T11:42:43.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T11:42:43.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.179+0000 7f244f21b700 1 Processor -- start 2023-10-06T11:42:43.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.179+0000 7f244f21b700 1 -- start start 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244f21b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24480675c0 0x7f24480679f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244f21b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2448159ba0 con 0x7f24480b27f0 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244f21b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24480cd320 con 0x7f24480cc7d0 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244f21b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24480b1320 con 0x7f24480675c0 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f2447fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24480b27f0 0x7f24480cda90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49610/0 (socket says 172.21.15.16:49610) 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244d7b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24480675c0 0x7f24480679f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f2447fff700 1 -- 172.21.15.16:0/884506176 learned_addr learned my addr 172.21.15.16:0/884506176 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f24477fe700 1 -- 172.21.15.16:0/884506176 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458361831 0 0) 0x7f2448159ba0 con 0x7f24480b27f0 2023-10-06T11:42:43.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f24477fe700 1 -- 172.21.15.16:0/884506176 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2438003570 con 0x7f24480b27f0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244d7b8700 1 -- 172.21.15.16:0/884506176 >> v1:172.21.15.177:6789/0 conn(0x7f24480cc7d0 legacy=0x7f24480b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244d7b8700 1 -- 172.21.15.16:0/884506176 >> v1:172.21.15.16:6789/0 conn(0x7f24480b27f0 legacy=0x7f24480cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244d7b8700 1 -- 172.21.15.16:0/884506176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480cd600 con 0x7f24480675c0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.180+0000 7f244d7b8700 1 --2- 172.21.15.16:0/884506176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480675c0 0x7f24480679f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2440009e60 tx=0x7f2440009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846db08be90ebe34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f24477fe700 1 -- 172.21.15.16:0/884506176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244000e070 con 0x7f24480675c0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f24477fe700 1 -- 172.21.15.16:0/884506176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244000bb20 con 0x7f24480675c0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f24477fe700 1 -- 172.21.15.16:0/884506176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440012430 con 0x7f24480675c0 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 -- 172.21.15.16:0/884506176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480675c0 msgr2=0x7f24480679f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 --2- 172.21.15.16:0/884506176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480675c0 0x7f24480679f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2440009e60 tx=0x7f2440009780 comp rx=0 tx=0).stop 2023-10-06T11:42:43.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 -- 172.21.15.16:0/884506176 shutdown_connections 2023-10-06T11:42:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 --2- 172.21.15.16:0/884506176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480675c0 0x7f24480679f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 -- 172.21.15.16:0/884506176 >> 172.21.15.16:0/884506176 conn(0x7f244806d520 msgr2=0x7f244806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 -- 172.21.15.16:0/884506176 shutdown_connections 2023-10-06T11:42:43.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.181+0000 7f244f21b700 1 -- 172.21.15.16:0/884506176 wait complete. 2023-10-06T11:42:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.182+0000 7f244f21b700 1 Processor -- start 2023-10-06T11:42:43.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.182+0000 7f244f21b700 1 -- start start 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 0x7f24481456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 0x7f2448158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480af2f0 con 0x7f24480b27f0 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244815a7a0 con 0x7f24480675c0 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244f21b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244815aa80 con 0x7f24480cc7d0 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f2447fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f2447fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60590/0 (socket says 172.21.15.16:60590) 2023-10-06T11:42:43.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f2447fff700 1 -- 172.21.15.16:0/1264365631 learned_addr learned my addr 172.21.15.16:0/1264365631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244cfb7700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 0x7f24481456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.183+0000 7f244d7b8700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 0x7f2448158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 -- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 msgr2=0x7f2448158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 0x7f2448158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 -- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 msgr2=0x7f24481456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 0x7f24481456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 -- 172.21.15.16:0/1264365631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440009a20 con 0x7f24480b27f0 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f244d7b8700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 0x7f2448158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:43.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f244cfb7700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 0x7f24481456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2447fff700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f243400c7c0 tx=0x7f2434007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434005b30 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434005c90 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244815ad60 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434007340 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.184+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448142e60 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.185+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448063e70 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.185+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f243401b400 con 0x7f24480b27f0 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.186+0000 7f2444ff9700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 0x7f242c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.186+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2434083ef0 con 0x7f24480b27f0 2023-10-06T11:42:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.186+0000 7f244cfb7700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 0x7f242c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.186+0000 7f244cfb7700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 0x7f242c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243c002880 tx=0x7f243c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.190+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2434051330 con 0x7f24480b27f0 2023-10-06T11:42:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.384+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f244806b4d0 con 0x7f24480b27f0 2023-10-06T11:42:43.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.708+0000 7f2444ff9700 1 -- 172.21.15.16:0/1264365631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2434054950 con 0x7f24480b27f0 2023-10-06T11:42:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 msgr2=0x7f242c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 0x7f242c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f243c002880 tx=0x7f243c009040 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 msgr2=0x7f2448158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f243400c7c0 tx=0x7f2434007e00 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 shutdown_connections 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f242c066f30 0x7f242c0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24480cc7d0 0x7f2448158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24480675c0 0x7f24481456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 --2- 172.21.15.16:0/1264365631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24480b27f0 0x7f2448158a10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.713+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 >> 172.21.15.16:0/1264365631 conn(0x7f244806d520 msgr2=0x7f24480cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.714+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 shutdown_connections 2023-10-06T11:42:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.714+0000 7f244f21b700 1 -- 172.21.15.16:0/1264365631 wait complete. 2023-10-06T11:42:43.717 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-10-06T11:42:43.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T11:42:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.854+0000 7fd51129c700 1 Processor -- start 2023-10-06T11:42:43.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.854+0000 7fd51129c700 1 -- start start 2023-10-06T11:42:43.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd51129c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd50c1570d0 0x7fd50c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd51129c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c06a7f0 con 0x7fd50c06cb20 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd51129c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c15a130 con 0x7fd50c0b3000 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd51129c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c15a410 con 0x7fd50c1570d0 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd50b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd50c06cb20 0x7fd50c06aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49614/0 (socket says 172.21.15.16:49614) 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd50b7fe700 1 -- 172.21.15.16:0/1853967856 learned_addr learned my addr 172.21.15.16:0/1853967856 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd50affd700 1 --2- 172.21.15.16:0/1853967856 >> v2:172.21.15.177:3300/0 conn(0x7fd50c1570d0 0x7fd50c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111515445 0 0) 0x7fd50c06a7f0 con 0x7fd50c06cb20 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f8003570 con 0x7fd50c06cb20 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1312638733 0 0) 0x7fd4f8003570 con 0x7fd50c06cb20 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 >> v2:172.21.15.177:3300/0 conn(0x7fd50c1570d0 msgr2=0x7fd50c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:43.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 --2- 172.21.15.16:0/1853967856 >> v2:172.21.15.177:3300/0 conn(0x7fd50c1570d0 0x7fd50c0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 >> v1:172.21.15.177:6789/0 conn(0x7fd50c0b3000 legacy=0x7fd50c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c15a6f0 con 0x7fd50c06cb20 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fd5040017f0 con 0x7fd50c06cb20 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 >> v1:172.21.15.16:6789/0 conn(0x7fd50c06cb20 legacy=0x7fd50c06aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4f8003a60 0x7fd4f8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 0x7fd4f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c15a410 con 0x7fd4f80051c0 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800c0e0 con 0x7fd4f8004810 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.855+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800c2a0 con 0x7fd4f8003a60 2023-10-06T11:42:43.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 0x7fd4f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50affd700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4f8003a60 0x7fd4f8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50a7fc700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 -- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4f8003a60 msgr2=0x7fd4f8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4f8003a60 0x7fd4f8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 -- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f8004810 msgr2=0x7fd4f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 -- 172.21.15.16:0/1853967856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8003570 con 0x7fd4f80051c0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50a7fc700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f8004810 0x7fd4f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 -- 172.21.15.16:0/1853967856 reap_dead start 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd50b7fe700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 0x7fd4f800ba60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd50400a920 tx=0x7fd5040021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ac9d02c64162d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040070d0 con 0x7fd4f80051c0 2023-10-06T11:42:43.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504007230 con 0x7fd4f80051c0 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.856+0000 7fd509ffb700 1 -- 172.21.15.16:0/1853967856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5040075d0 con 0x7fd4f80051c0 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 -- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 msgr2=0x7fd4f800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 0x7fd4f800ba60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd50400a920 tx=0x7fd5040021f0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 -- 172.21.15.16:0/1853967856 shutdown_connections 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 --2- 172.21.15.16:0/1853967856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f80051c0 0x7fd4f800ba60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 -- 172.21.15.16:0/1853967856 >> 172.21.15.16:0/1853967856 conn(0x7fd50c068030 msgr2=0x7fd50c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 -- 172.21.15.16:0/1853967856 shutdown_connections 2023-10-06T11:42:43.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.857+0000 7fd51129c700 1 -- 172.21.15.16:0/1853967856 wait complete. 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.858+0000 7fd51129c700 1 Processor -- start 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.858+0000 7fd51129c700 1 -- start start 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f80051c0 0x7fd50c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd50c0a98a0 0x7fd50c0ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c14f5f0 con 0x7fd50c14ed40 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0afb90 con 0x7fd4f80051c0 2023-10-06T11:42:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd51129c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c157f50 con 0x7fd50c0a98a0 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60618/0 (socket says 172.21.15.16:60618) 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 -- 172.21.15.16:0/1847816106 learned_addr learned my addr 172.21.15.16:0/1847816106 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50affd700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f80051c0 0x7fd50c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50b7fe700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd50c0a98a0 0x7fd50c0ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 -- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd50c0a98a0 msgr2=0x7fd50c0ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd50c0a98a0 0x7fd50c0ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 -- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f80051c0 msgr2=0x7fd50c0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f80051c0 0x7fd50c0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 -- 172.21.15.16:0/1847816106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c158230 con 0x7fd50c14ed40 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd50a7fc700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400e800 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.859+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4011460 con 0x7fd50c14ed40 2023-10-06T11:42:43.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.860+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4010040 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.860+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40117d0 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.860+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c158510 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.860+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c158ad0 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.861+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c0b2e10 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.861+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd4f4011b80 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.861+0000 7fd502ffd700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 0x7fd4ec069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.862+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fd4f4083800 con 0x7fd50c14ed40 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.862+0000 7fd50affd700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 0x7fd4ec069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:43.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.862+0000 7fd50affd700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 0x7fd4ec069410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc006f00 tx=0x7fd4fc0091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:43.866+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4f404d5a0 con 0x7fd50c14ed40 2023-10-06T11:42:44.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.060+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd50c158cf0 con 0x7fd50c14ed40 2023-10-06T11:42:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.714+0000 7fd502ffd700 1 -- 172.21.15.16:0/1847816106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd4f40190d0 con 0x7fd50c14ed40 2023-10-06T11:42:44.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 msgr2=0x7fd4ec069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 0x7fd4ec069410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc006f00 tx=0x7fd4fc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 msgr2=0x7fd50c14e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400e800 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 shutdown_connections 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd4ec066f60 0x7fd4ec069410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd50c0a98a0 0x7fd50c0ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd4f80051c0 0x7fd50c0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 --2- 172.21.15.16:0/1847816106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd50c14ed40 0x7fd50c14e510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.724+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 >> 172.21.15.16:0/1847816106 conn(0x7fd50c068030 msgr2=0x7fd50c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:44.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.726+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 shutdown_connections 2023-10-06T11:42:44.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.727+0000 7fd51129c700 1 -- 172.21.15.16:0/1847816106 wait complete. 2023-10-06T11:42:44.731 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-10-06T11:42:44.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T11:42:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.869+0000 7fd031b0f700 1 Processor -- start 2023-10-06T11:42:44.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.869+0000 7fd031b0f700 1 -- start start 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd031b0f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd02c0cb440 0x7fd02c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd031b0f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd02c1432c0 0x7fd02c0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd031b0f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c15dd40 con 0x7fd02c142ef0 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd031b0f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c06e530 con 0x7fd02c0cb440 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd031b0f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c15ac80 con 0x7fd02c1432c0 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd030b0d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd02c142ef0 0x7fd02c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49616/0 (socket says 172.21.15.16:49616) 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd02c0cb440 0x7fd02c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd030b0d700 1 -- 172.21.15.16:0/598829013 learned_addr learned my addr 172.21.15.16:0/598829013 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd03130e700 1 --2- 172.21.15.16:0/598829013 >> v2:172.21.15.177:3300/0 conn(0x7fd02c1432c0 0x7fd02c0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02bfff700 1 -- 172.21.15.16:0/598829013 >> v2:172.21.15.177:3300/0 conn(0x7fd02c1432c0 msgr2=0x7fd02c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02bfff700 1 --2- 172.21.15.16:0/598829013 >> v2:172.21.15.177:3300/0 conn(0x7fd02c1432c0 0x7fd02c0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02bfff700 1 -- 172.21.15.16:0/598829013 >> v1:172.21.15.16:6789/0 conn(0x7fd02c142ef0 legacy=0x7fd02c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02bfff700 1 -- 172.21.15.16:0/598829013 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0cb8c0 con 0x7fd02c0cb440 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.870+0000 7fd02b7fe700 1 -- 172.21.15.16:0/598829013 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659314233 0 0) 0x7fd02c15dd40 con 0x7fd02c142ef0 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd02bfff700 1 --2- 172.21.15.16:0/598829013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c0cb440 0x7fd02c0cbf40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098e0 tx=0x7fd0180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a46e48a7f8a7e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd02b7fe700 1 -- 172.21.15.16:0/598829013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018011630 con 0x7fd02c0cb440 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd02b7fe700 1 -- 172.21.15.16:0/598829013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018011790 con 0x7fd02c0cb440 2023-10-06T11:42:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd02b7fe700 1 -- 172.21.15.16:0/598829013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018011b30 con 0x7fd02c0cb440 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 -- 172.21.15.16:0/598829013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c0cb440 msgr2=0x7fd02c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 --2- 172.21.15.16:0/598829013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c0cb440 0x7fd02c0cbf40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098e0 tx=0x7fd0180091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 -- 172.21.15.16:0/598829013 shutdown_connections 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 --2- 172.21.15.16:0/598829013 >> v2:172.21.15.177:3300/0 conn(0x7fd02c1432c0 0x7fd02c0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 --2- 172.21.15.16:0/598829013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c0cb440 0x7fd02c0cbf40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 -- 172.21.15.16:0/598829013 >> 172.21.15.16:0/598829013 conn(0x7fd02c06cee0 msgr2=0x7fd02c067c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 -- 172.21.15.16:0/598829013 shutdown_connections 2023-10-06T11:42:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.871+0000 7fd031b0f700 1 -- 172.21.15.16:0/598829013 wait complete. 2023-10-06T11:42:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.872+0000 7fd031b0f700 1 Processor -- start 2023-10-06T11:42:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 -- start start 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd02c142ef0 0x7fd02c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 0x7fd02c0add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0b3210 con 0x7fd02c1432c0 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0b0a60 con 0x7fd02c142ef0 2023-10-06T11:42:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd031b0f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0b2600 con 0x7fd02c0b3c20 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.873+0000 7fd02bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60636/0 (socket says 172.21.15.16:60636) 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 -- 172.21.15.16:0/3809564725 learned_addr learned my addr 172.21.15.16:0/3809564725 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd030b0d700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd02c142ef0 0x7fd02c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd03130e700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 0x7fd02c0add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 -- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 msgr2=0x7fd02c0add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 0x7fd02c0add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 -- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd02c142ef0 msgr2=0x7fd02c0b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd02c142ef0 0x7fd02c0b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 -- 172.21.15.16:0/3809564725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009420 con 0x7fd02c1432c0 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd03130e700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 0x7fd02c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd02bfff700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0180091e0 tx=0x7fd018017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018011b10 con 0x7fd02c1432c0 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0180253f0 con 0x7fd02c1432c0 2023-10-06T11:42:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c0b28e0 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180257d0 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.874+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c0b2ea0 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.875+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02c0b3380 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.876+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd018025c00 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.876+0000 7fd028ff9700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.876+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fd018085e30 con 0x7fd02c1432c0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.876+0000 7fd030b0d700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.877+0000 7fd030b0d700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd020007fd0 tx=0x7fd020007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:44.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:44.881+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd018052fa0 con 0x7fd02c1432c0 2023-10-06T11:42:45.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.118+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd02c04bb40 con 0x7fd02c1432c0 2023-10-06T11:42:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.717+0000 7fd028ff9700 1 -- 172.21.15.16:0/3809564725 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd0180565c0 con 0x7fd02c1432c0 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 msgr2=0x7fd0100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd020007fd0 tx=0x7fd020007d30 comp rx=0 tx=0).stop 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 msgr2=0x7fd02c0b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd0180091e0 tx=0x7fd018017000 comp rx=0 tx=0).stop 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 shutdown_connections 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd02c0b3c20 0x7fd02c0add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd02c142ef0 0x7fd02c0b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 --2- 172.21.15.16:0/3809564725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd02c1432c0 0x7fd02c0b1db0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 >> 172.21.15.16:0/3809564725 conn(0x7fd02c06cee0 msgr2=0x7fd02c1415c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 shutdown_connections 2023-10-06T11:42:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.722+0000 7fd031b0f700 1 -- 172.21.15.16:0/3809564725 wait complete. 2023-10-06T11:42:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T11:42:45.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.869+0000 7fb91e7b6700 1 Processor -- start 2023-10-06T11:42:45.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.869+0000 7fb91e7b6700 1 -- start start 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91e7b6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9180cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91e7b6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9180cd6a0 0x7fb9180ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91e7b6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9180cdb10 con 0x7fb9180cf280 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91e7b6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9180b3110 con 0x7fb9180cd6a0 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91e7b6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180ccb20 con 0x7fb91815cc20 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91cd53700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9180cf280 0x7fb91815d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49632/0 (socket says 172.21.15.16:49632) 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.870+0000 7fb91cd53700 1 -- 172.21.15.16:0/4148826332 learned_addr learned my addr 172.21.15.16:0/4148826332 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 --2- 172.21.15.16:0/4148826332 >> v2:172.21.15.177:3300/0 conn(0x7fb9180cd6a0 0x7fb9180ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb917fff700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9180cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582247926 0 0) 0x7fb9180cdb10 con 0x7fb9180cf280 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb904003570 con 0x7fb9180cf280 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638615909 0 0) 0x7fb904003570 con 0x7fb9180cf280 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 msgr2=0x7fb9180cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9180cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 >> v2:172.21.15.177:3300/0 conn(0x7fb9180cd6a0 msgr2=0x7fb9180ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 --2- 172.21.15.16:0/4148826332 >> v2:172.21.15.177:3300/0 conn(0x7fb9180cd6a0 0x7fb9180ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180cbf10 con 0x7fb9180cf280 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fb9100017f0 con 0x7fb9180cf280 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 >> v1:172.21.15.16:6789/0 conn(0x7fb9180cf280 legacy=0x7fb91815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904003a60 0x7fb904003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904004810 0x7fb904004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180b3110 con 0x7fb904004810 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180ccb20 con 0x7fb904003a60 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90400bae0 con 0x7fb91815cc20 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904004810 0x7fb904004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb917fff700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904003a60 0x7fb904003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb91cd53700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 -- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 msgr2=0x7fb9040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 -- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904003a60 msgr2=0x7fb904003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:45.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904003a60 0x7fb904003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb9177fe700 1 -- 172.21.15.16:0/4148826332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904003570 con 0x7fb904004810 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.871+0000 7fb91cd53700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb91815cc20 0x7fb9040051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb9177fe700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904004810 0x7fb904004c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb900009c00 tx=0x7fb9000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fb5e3729e319cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000e710 con 0x7fb904004810 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90000e870 con 0x7fb904004810 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb916ffd700 1 -- 172.21.15.16:0/4148826332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000ec30 con 0x7fb904004810 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 -- 172.21.15.16:0/4148826332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904004810 msgr2=0x7fb904004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/4148826332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb904004810 0x7fb904004c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb900009c00 tx=0x7fb9000091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91cd53700 1 -- 172.21.15.16:0/4148826332 reap_dead start 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 -- 172.21.15.16:0/4148826332 shutdown_connections 2023-10-06T11:42:45.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 -- 172.21.15.16:0/4148826332 >> 172.21.15.16:0/4148826332 conn(0x7fb918099b40 msgr2=0x7fb91806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 -- 172.21.15.16:0/4148826332 shutdown_connections 2023-10-06T11:42:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.872+0000 7fb91e7b6700 1 -- 172.21.15.16:0/4148826332 wait complete. 2023-10-06T11:42:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.873+0000 7fb91e7b6700 1 Processor -- start 2023-10-06T11:42:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 -- start start 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904004810 0x7fb918143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb918156f70 0x7fb9180b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91816d6a0 con 0x7fb91816ccc0 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91815b150 con 0x7fb904004810 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91e7b6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9180c79b0 con 0x7fb918156f70 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.874+0000 7fb91cd53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60662/0 (socket says 172.21.15.16:60662) 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 -- 172.21.15.16:0/2121752641 learned_addr learned my addr 172.21.15.16:0/2121752641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb917fff700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904004810 0x7fb918143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb9177fe700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb918156f70 0x7fb9180b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 -- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb918156f70 msgr2=0x7fb9180b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb918156f70 0x7fb9180b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 -- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904004810 msgr2=0x7fb918143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904004810 0x7fb918143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 -- 172.21.15.16:0/2121752641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180c7c90 con 0x7fb91816ccc0 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91cd53700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb910000f80 tx=0x7fb91000e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910007b60 con 0x7fb91816ccc0 2023-10-06T11:42:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91000eb80 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910013420 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91815ade0 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.875+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9180c7f70 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.876+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91815b2c0 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.877+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb910013790 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.877+0000 7fb90ffff700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 0x7fb8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.877+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb910084b90 con 0x7fb91816ccc0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.877+0000 7fb917fff700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 0x7fb8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:45.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.878+0000 7fb917fff700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 0x7fb8f80693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb908006f00 tx=0x7fb9080091a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:45.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:45.881+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb910051ca0 con 0x7fb91816ccc0 2023-10-06T11:42:46.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.076+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb91804c380 con 0x7fb91816ccc0 2023-10-06T11:42:46.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.077+0000 7fb90ffff700 1 -- 172.21.15.16:0/2121752641 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16879 (secure 0 0 0) 0x7fb9100552c0 con 0x7fb91816ccc0 2023-10-06T11:42:46.079 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:46.079 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:45.120536+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:46.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 msgr2=0x7fb8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 0x7fb8f80693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb908006f00 tx=0x7fb9080091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 msgr2=0x7fb91816d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb910000f80 tx=0x7fb91000e440 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 shutdown_connections 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb91816ccc0 0x7fb91816d160 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8f8066f30 0x7fb8f80693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb904004810 0x7fb918143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 --2- 172.21.15.16:0/2121752641 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb918156f70 0x7fb9180b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 >> 172.21.15.16:0/2121752641 conn(0x7fb918099b40 msgr2=0x7fb9180c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 shutdown_connections 2023-10-06T11:42:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.079+0000 7fb91e7b6700 1 -- 172.21.15.16:0/2121752641 wait complete. 2023-10-06T11:42:46.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 Processor -- start 2023-10-06T11:42:46.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 -- start start 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 0x7f7ad40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ad40cd6a0 0x7f7ad40ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad40cdb10 con 0x7f7ad40cf280 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40b3110 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.211+0000 7f7ada9a4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40ccb20 con 0x7f7ad415cc20 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ad40cd6a0 0x7f7ad40ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad8f41700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ad40cf280 0x7f7ad415d390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49646/0 (socket says 172.21.15.16:49646) 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ad40cd6a0 0x7f7ad40ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60668/0 (socket says 172.21.15.16:60668) 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad8f41700 1 -- 172.21.15.16:0/2981746071 learned_addr learned my addr 172.21.15.16:0/2981746071 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/2981746071 >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 0x7f7ad40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 -- 172.21.15.16:0/2981746071 >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 msgr2=0x7f7ad40cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 --2- 172.21.15.16:0/2981746071 >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 0x7f7ad40cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 -- 172.21.15.16:0/2981746071 >> v1:172.21.15.16:6789/0 conn(0x7f7ad40cf280 legacy=0x7f7ad415d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad2ffd700 1 -- 172.21.15.16:0/2981746071 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63771849 0 0) 0x7f7ad40cdb10 con 0x7f7ad40cf280 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 -- 172.21.15.16:0/2981746071 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40cbf10 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/2981746071 >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 0x7f7ad40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad37fe700 1 --2- 172.21.15.16:0/2981746071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad40ced40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4089130 tx=0x7f7ac80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45bfabc13e76420 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad2ffd700 1 -- 172.21.15.16:0/2981746071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800b6b0 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad2ffd700 1 -- 172.21.15.16:0/2981746071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac800b810 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.212+0000 7f7ad2ffd700 1 -- 172.21.15.16:0/2981746071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800bbb0 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 -- 172.21.15.16:0/2981746071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad40cd6a0 msgr2=0x7f7ad40ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/2981746071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad40ced40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4089130 tx=0x7f7ac80091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 -- 172.21.15.16:0/2981746071 shutdown_connections 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/2981746071 >> v2:172.21.15.177:3300/0 conn(0x7f7ad415cc20 0x7f7ad40cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/2981746071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad40ced40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 -- 172.21.15.16:0/2981746071 >> 172.21.15.16:0/2981746071 conn(0x7f7ad4099b40 msgr2=0x7f7ad406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:46.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 -- 172.21.15.16:0/2981746071 shutdown_connections 2023-10-06T11:42:46.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.213+0000 7f7ada9a4700 1 -- 172.21.15.16:0/2981746071 wait complete. 2023-10-06T11:42:46.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.214+0000 7f7ada9a4700 1 Processor -- start 2023-10-06T11:42:46.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.214+0000 7f7ada9a4700 1 -- start start 2023-10-06T11:42:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad416a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ad40cf280 0x7f7ad406f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4158010 con 0x7f7ad415cc20 2023-10-06T11:42:46.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4169c50 con 0x7f7ad40cd6a0 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ada9a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b61a0 con 0x7f7ad40cf280 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60678/0 (socket says 172.21.15.16:60678) 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 -- 172.21.15.16:0/1367704095 learned_addr learned my addr 172.21.15.16:0/1367704095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad416a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad37fe700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ad40cf280 0x7f7ad406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 -- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ad40cf280 msgr2=0x7f7ad406f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ad40cf280 0x7f7ad406f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 -- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 msgr2=0x7f7ad416a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad416a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad8f41700 1 -- 172.21.15.16:0/1367704095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8009420 con 0x7f7ad415cc20 2023-10-06T11:42:46.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.215+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad416a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7ad8f41700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00cc20 tx=0x7f7acc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc004bb0 con 0x7f7ad415cc20 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc004d10 con 0x7f7ad415cc20 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00abd0 con 0x7f7ad415cc20 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad4157800 con 0x7f7ad415cc20 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.216+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad4157d30 con 0x7f7ad415cc20 2023-10-06T11:42:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.217+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad404c380 con 0x7f7ad415cc20 2023-10-06T11:42:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.217+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7acc0070f0 con 0x7f7ad415cc20 2023-10-06T11:42:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.217+0000 7f7abbfff700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 0x7f7ac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.217+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7acc0844c0 con 0x7f7ad415cc20 2023-10-06T11:42:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.218+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 0x7f7ac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.218+0000 7f7ad3fff700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 0x7f7ac00693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b26b0 tx=0x7f7ac4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.222+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7acc051650 con 0x7f7ad415cc20 2023-10-06T11:42:46.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.415+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad40cc1f0 con 0x7f7ad415cc20 2023-10-06T11:42:46.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.416+0000 7f7abbfff700 1 -- 172.21.15.16:0/1367704095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16879 (secure 0 0 0) 0x7f7acc018090 con 0x7f7ad415cc20 2023-10-06T11:42:46.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:46.419 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":31,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:45.120536+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 msgr2=0x7f7ac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 0x7f7ac00693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b26b0 tx=0x7f7ac4008040 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 msgr2=0x7f7ad40b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00cc20 tx=0x7f7acc0019b0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 shutdown_connections 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad415cc20 0x7f7ad40b54c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7ac0066f30 0x7f7ac00693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ad40cd6a0 0x7f7ad416a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 --2- 172.21.15.16:0/1367704095 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ad40cf280 0x7f7ad406f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 >> 172.21.15.16:0/1367704095 conn(0x7f7ad4099b40 msgr2=0x7f7ad4154490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 shutdown_connections 2023-10-06T11:42:46.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.418+0000 7f7ada9a4700 1 -- 172.21.15.16:0/1367704095 wait complete. 2023-10-06T11:42:46.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T11:42:46.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.553+0000 7f014a112700 1 Processor -- start 2023-10-06T11:42:46.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.553+0000 7f014a112700 1 -- start start 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f014a112700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01440cda10 0x7f01440b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f014a112700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0144159b10 0x7f01440cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f014a112700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01440cead0 con 0x7f0144159740 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f014a112700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01440affc0 con 0x7f0144159b10 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f014a112700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01440cdec0 con 0x7f01440cda10 2023-10-06T11:42:46.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f0143fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0144159b10 0x7f01440cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f0143fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0144159b10 0x7f01440cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60694/0 (socket says 172.21.15.16:60694) 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.554+0000 7f0143fff700 1 -- 172.21.15.16:0/3882278592 learned_addr learned my addr 172.21.15.16:0/3882278592 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0142ffd700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01440cda10 0x7f01440b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0143fff700 1 -- 172.21.15.16:0/3882278592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01440cda10 msgr2=0x7f01440b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0143fff700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01440cda10 0x7f01440b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0143fff700 1 -- 172.21.15.16:0/3882278592 >> v1:172.21.15.177:6789/0 conn(0x7f0144159740 legacy=0x7f01440b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0143fff700 1 -- 172.21.15.16:0/3882278592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440ce1a0 con 0x7f0144159b10 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f0143fff700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01440cef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f013c009f20 tx=0x7f013c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb494352a0af882 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f01427fc700 1 -- 172.21.15.16:0/3882278592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00e070 con 0x7f0144159b10 2023-10-06T11:42:46.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f01427fc700 1 -- 172.21.15.16:0/3882278592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00ba40 con 0x7f0144159b10 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.555+0000 7f01427fc700 1 -- 172.21.15.16:0/3882278592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c0092e0 con 0x7f0144159b10 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 -- 172.21.15.16:0/3882278592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 msgr2=0x7f01440cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01440cef60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f013c009f20 tx=0x7f013c009840 comp rx=0 tx=0).stop 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 -- 172.21.15.16:0/3882278592 shutdown_connections 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01440cef60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 --2- 172.21.15.16:0/3882278592 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01440cda10 0x7f01440b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 -- 172.21.15.16:0/3882278592 >> 172.21.15.16:0/3882278592 conn(0x7f0144099a40 msgr2=0x7f014406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 -- 172.21.15.16:0/3882278592 shutdown_connections 2023-10-06T11:42:46.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.556+0000 7f014a112700 1 -- 172.21.15.16:0/3882278592 wait complete. 2023-10-06T11:42:46.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.557+0000 7f014a112700 1 Processor -- start 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.557+0000 7f014a112700 1 -- start start 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 0x7f0144153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 0x7f0144152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440aaa30 con 0x7f0144159b10 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441535f0 con 0x7f01440cda10 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01441541e0 con 0x7f0144159740 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60708/0 (socket says 172.21.15.16:60708) 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 -- 172.21.15.16:0/3143500395 learned_addr learned my addr 172.21.15.16:0/3143500395 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:46.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0142ffd700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 0x7f0144152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f01437fe700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 0x7f0144153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 -- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 msgr2=0x7f0144152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 0x7f0144152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 -- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 msgr2=0x7f0144153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 0x7f0144153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 -- 172.21.15.16:0/3143500395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013c009ae0 con 0x7f0144159b10 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0142ffd700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 0x7f0144152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f01437fe700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 0x7f0144153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f0143fff700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f014406d350 tx=0x7f013c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c0094e0 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c0032f0 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0144155f40 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c01d650 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.558+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014415a330 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.560+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144156460 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.560+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f013c01da80 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.560+0000 7f012b7fe700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 0x7f01300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.560+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f013c085f80 con 0x7f0144159b10 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.561+0000 7f01437fe700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 0x7f01300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:46.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.561+0000 7f01437fe700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 0x7f01300693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0134002880 tx=0x7f0134009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:46.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.565+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f013c0b75a0 con 0x7f0144159b10 2023-10-06T11:42:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:46.805+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f014404c380 con 0x7f0144159b10 2023-10-06T11:42:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.742+0000 7f012b7fe700 1 -- 172.21.15.16:0/3143500395 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0144156460 con 0x7f0144159b10 2023-10-06T11:42:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 msgr2=0x7f01300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 0x7f01300693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0134002880 tx=0x7f0134009040 comp rx=0 tx=0).stop 2023-10-06T11:42:47.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 msgr2=0x7f01441530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f014406d350 tx=0x7f013c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 shutdown_connections 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0144159b10 0x7f01441530b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0130066f30 0x7f01300693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01440cda10 0x7f0144153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 --2- 172.21.15.16:0/3143500395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0144159740 0x7f0144152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 >> 172.21.15.16:0/3143500395 conn(0x7f0144099a40 msgr2=0x7f01441587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 shutdown_connections 2023-10-06T11:42:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.745+0000 7f014a112700 1 -- 172.21.15.16:0/3143500395 wait complete. 2023-10-06T11:42:47.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T11:42:47.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.887+0000 7f4bc0bb3700 1 Processor -- start 2023-10-06T11:42:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.887+0000 7f4bc0bb3700 1 -- start start 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.888+0000 7f4bc0bb3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4bbc155c90 0x7f4bbc15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.888+0000 7f4bc0bb3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.888+0000 7f4bc0bb3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc15a920 con 0x7f4bbc0ac400 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.888+0000 7f4bc0bb3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bbc0ac900 con 0x7f4bbc155c90 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.888+0000 7f4bc0bb3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0acbe0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:36614/0 (socket says 172.21.15.16:36614) 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 -- 172.21.15.16:0/2308558579 learned_addr learned my addr 172.21.15.16:0/2308558579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bba59c700 1 --2- 172.21.15.16:0/2308558579 >> v2:172.21.15.177:3300/0 conn(0x7f4bbc155c90 0x7f4bbc15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 -- 172.21.15.16:0/2308558579 >> v2:172.21.15.177:3300/0 conn(0x7f4bbc155c90 msgr2=0x7f4bbc15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 --2- 172.21.15.16:0/2308558579 >> v2:172.21.15.177:3300/0 conn(0x7f4bbc155c90 0x7f4bbc15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 -- 172.21.15.16:0/2308558579 >> v1:172.21.15.177:6789/0 conn(0x7f4bbc0ac400 legacy=0x7f4bbc0c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 -- 172.21.15.16:0/2308558579 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0acec0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bbad9d700 1 --2- 172.21.15.16:0/2308558579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40099b0 tx=0x7f4bb4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823463857214248b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.889+0000 7f4bb959a700 1 -- 172.21.15.16:0/2308558579 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4009ea0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bb959a700 1 -- 172.21.15.16:0/2308558579 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb400b860 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bb959a700 1 -- 172.21.15.16:0/2308558579 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400bc00 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/2308558579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 msgr2=0x7f4bbc06aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/2308558579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bb40099b0 tx=0x7f4bb4009250 comp rx=0 tx=0).stop 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/2308558579 shutdown_connections 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/2308558579 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0c93a0 0x7f4bbc06aec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/2308558579 >> v2:172.21.15.177:3300/0 conn(0x7f4bbc155c90 0x7f4bbc15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/2308558579 >> 172.21.15.16:0/2308558579 conn(0x7f4bbc06ce90 msgr2=0x7f4bbc0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/2308558579 shutdown_connections 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.890+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/2308558579 wait complete. 2023-10-06T11:42:47.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.891+0000 7f4bc0bb3700 1 Processor -- start 2023-10-06T11:42:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.891+0000 7f4bc0bb3700 1 -- start start 2023-10-06T11:42:47.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0ac400 0x7f4bbc14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bbc155c90 0x7f4bbc152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc167f90 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc153420 con 0x7f4bbc155c90 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bc0bb3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc14f6a0 con 0x7f4bbc0ac400 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bb9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bb9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60712/0 (socket says 172.21.15.16:60712) 2023-10-06T11:42:47.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bb9d9b700 1 -- 172.21.15.16:0/3321050073 learned_addr learned my addr 172.21.15.16:0/3321050073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bba59c700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0ac400 0x7f4bbc14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.892+0000 7f4bbad9d700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bbc155c90 0x7f4bbc152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 -- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0ac400 msgr2=0x7f4bbc14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0ac400 0x7f4bbc14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 -- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bbc155c90 msgr2=0x7f4bbc152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bbc155c90 0x7f4bbc152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 -- 172.21.15.16:0/3321050073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb40094f0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb9d9b700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400c7c0 tx=0x7f4ba4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4011590 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba40116f0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc167940 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4011ab0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.893+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc14c7d0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.894+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc04c380 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.895+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ba401b3b0 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.895+0000 7f4bb27fc700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 0x7f4b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.895+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ba4083e70 con 0x7f4bbc0c93a0 2023-10-06T11:42:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.896+0000 7f4bba59c700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 0x7f4b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.896+0000 7f4bba59c700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 0x7f4b9c0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009fd0 tx=0x7f4bac00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:47.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:47.899+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ba4051000 con 0x7f4bbc0c93a0 2023-10-06T11:42:48.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.137+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4bbc0c9b70 con 0x7f4bbc0c93a0 2023-10-06T11:42:48.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.744+0000 7f4bb27fc700 1 -- 172.21.15.16:0/3321050073 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ba4017020 con 0x7f4bbc0c93a0 2023-10-06T11:42:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 msgr2=0x7f4b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 0x7f4b9c0693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009fd0 tx=0x7f4bac00b040 comp rx=0 tx=0).stop 2023-10-06T11:42:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 msgr2=0x7f4bbc1529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400c7c0 tx=0x7f4ba4007f50 comp rx=0 tx=0).stop 2023-10-06T11:42:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 shutdown_connections 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4b9c066f30 0x7f4b9c0693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bbc155c90 0x7f4bbc152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bbc0ac400 0x7f4bbc14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 --2- 172.21.15.16:0/3321050073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4bbc0c93a0 0x7f4bbc1529a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 >> 172.21.15.16:0/3321050073 conn(0x7f4bbc06ce90 msgr2=0x7f4bbc0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 shutdown_connections 2023-10-06T11:42:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.747+0000 7f4bc0bb3700 1 -- 172.21.15.16:0/3321050073 wait complete. 2023-10-06T11:42:48.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.887+0000 7fabb2735700 1 Processor -- start 2023-10-06T11:42:48.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.887+0000 7fabb2735700 1 -- start start 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb2735700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 0x7fabac06f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb2735700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac06ab00 con 0x7fabac06d960 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb2735700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0ac030 con 0x7fabac06eca0 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb2735700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0ac310 con 0x7fabac153f40 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb1f34700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fabac06d960 0x7fabac159db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36272/0 (socket says 172.21.15.16:36272) 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb1f34700 1 -- 172.21.15.16:0/1860573335 learned_addr learned my addr 172.21.15.16:0/1860573335 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:48.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fabb0f32700 1 --2- 172.21.15.16:0/1860573335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 0x7fabac06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814566365 0 0) 0x7fabac06ab00 con 0x7fabac06d960 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.888+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94003570 con 0x7fabac06d960 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163037742 0 0) 0x7fabac0ac030 con 0x7fabac06eca0 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabac06ab00 con 0x7fabac06eca0 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fabb0f32700 1 -- 172.21.15.16:0/1860573335 >> v1:172.21.15.177:6789/0 conn(0x7fabac06eca0 legacy=0x7fabac153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148486017 0 0) 0x7fab94003570 con 0x7fabac06d960 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fabb0f32700 1 -- 172.21.15.16:0/1860573335 >> v1:172.21.15.16:6789/0 conn(0x7fabac06d960 legacy=0x7fabac159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fabb0f32700 1 -- 172.21.15.16:0/1860573335 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac06a4b0 con 0x7fabac153f40 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fabb0f32700 1 --2- 172.21.15.16:0/1860573335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 0x7fabac06f070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faba0009940 tx=0x7faba00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70d75a3ad19644e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0017f0 con 0x7fabac153f40 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba0011420 con 0x7fabac153f40 2023-10-06T11:42:48.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.889+0000 7fababfff700 1 -- 172.21.15.16:0/1860573335 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba00117c0 con 0x7fabac153f40 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 -- 172.21.15.16:0/1860573335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 msgr2=0x7fabac06f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 --2- 172.21.15.16:0/1860573335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 0x7fabac06f070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faba0009940 tx=0x7faba00091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 -- 172.21.15.16:0/1860573335 shutdown_connections 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 --2- 172.21.15.16:0/1860573335 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac153f40 0x7fabac06f070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 -- 172.21.15.16:0/1860573335 >> 172.21.15.16:0/1860573335 conn(0x7fabac0ccab0 msgr2=0x7fabac0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 -- 172.21.15.16:0/1860573335 shutdown_connections 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.890+0000 7fabb2735700 1 -- 172.21.15.16:0/1860573335 wait complete. 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.891+0000 7fabb2735700 1 Processor -- start 2023-10-06T11:42:48.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.891+0000 7fabb2735700 1 -- start start 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac06d960 0x7fabac1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 0x7fabac15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac157420 con 0x7fabac153f40 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac157fa0 con 0x7fabac06eca0 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb2735700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac157670 con 0x7fabac06d960 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47528/0 (socket says 172.21.15.16:47528) 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 -- 172.21.15.16:0/3648978222 learned_addr learned my addr 172.21.15.16:0/3648978222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1733700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac06d960 0x7fabac1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:48.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb0f32700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 0x7fabac15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 -- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac06d960 msgr2=0x7fabac1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac06d960 0x7fabac1507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 -- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 msgr2=0x7fabac15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 0x7fabac15c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 -- 172.21.15.16:0/3648978222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba0009420 con 0x7fabac153f40 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb0f32700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 0x7fabac15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7fabb1f34700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00cca0 tx=0x7fab9c007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.892+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c004e90 con 0x7fabac153f40 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.893+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac157950 con 0x7fabac153f40 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.893+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c00a730 con 0x7fabac153f40 2023-10-06T11:42:48.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.893+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c00aad0 con 0x7fabac153f40 2023-10-06T11:42:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.893+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac150d20 con 0x7fabac153f40 2023-10-06T11:42:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.893+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac158130 con 0x7fabac153f40 2023-10-06T11:42:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.894+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab9c007150 con 0x7fabac153f40 2023-10-06T11:42:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.894+0000 7faba97fa700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:48.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.894+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fab9c083e20 con 0x7fabac153f40 2023-10-06T11:42:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.895+0000 7fabb1733700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.895+0000 7fabb1733700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 0x7fab980693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faba4007fd0 tx=0x7faba4007d50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:48.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:48.899+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab9c050f30 con 0x7fabac153f40 2023-10-06T11:42:49.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.135+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabac04c380 con 0x7fabac153f40 2023-10-06T11:42:49.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.136+0000 7faba97fa700 1 -- 172.21.15.16:0/3648978222 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7fab9c018090 con 0x7fabac153f40 2023-10-06T11:42:49.138 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:49.138 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 msgr2=0x7fab980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 0x7fab980693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7faba4007fd0 tx=0x7faba4007d50 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 msgr2=0x7fabac0cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fab9c00cca0 tx=0x7fab9c007d50 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 shutdown_connections 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabac153f40 0x7fabac0cce80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fab98066f30 0x7fab980693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabac06d960 0x7fabac1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 --2- 172.21.15.16:0/3648978222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabac06eca0 0x7fabac15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 >> 172.21.15.16:0/3648978222 conn(0x7fabac0ccab0 msgr2=0x7fabac151770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 shutdown_connections 2023-10-06T11:42:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.138+0000 7fabb2735700 1 -- 172.21.15.16:0/3648978222 wait complete. 2023-10-06T11:42:49.142 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-10-06T11:42:49.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:42:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.269+0000 7fc3da15c700 1 Processor -- start 2023-10-06T11:42:49.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.269+0000 7fc3da15c700 1 -- start start 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3da15c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3d40ca0c0 0x7fc3d40a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3da15c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc3d40c7370 0x7fc3d40c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3da15c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d406b430 con 0x7fc3d4159a10 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3da15c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40abc10 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3da15c700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40c7c80 con 0x7fc3d40c7370 2023-10-06T11:42:49.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3d37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3d4159a10 0x7fc3d406ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36280/0 (socket says 172.21.15.16:36280) 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3d37fe700 1 -- 172.21.15.16:0/333304153 learned_addr learned my addr 172.21.15.16:0/333304153 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3d2ffd700 1 --2- 172.21.15.16:0/333304153 >> v2:172.21.15.16:3300/0 conn(0x7fc3d40ca0c0 0x7fc3d40a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.270+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/333304153 >> v2:172.21.15.177:3300/0 conn(0x7fc3d40c7370 0x7fc3d40c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d27fc700 1 -- 172.21.15.16:0/333304153 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700012422 0 0) 0x7fc3d406b430 con 0x7fc3d4159a10 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d27fc700 1 -- 172.21.15.16:0/333304153 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3bc003570 con 0x7fc3d4159a10 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d2ffd700 1 -- 172.21.15.16:0/333304153 >> v2:172.21.15.177:3300/0 conn(0x7fc3d40c7370 msgr2=0x7fc3d40c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d2ffd700 1 --2- 172.21.15.16:0/333304153 >> v2:172.21.15.177:3300/0 conn(0x7fc3d40c7370 0x7fc3d40c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d2ffd700 1 -- 172.21.15.16:0/333304153 >> v1:172.21.15.16:6789/0 conn(0x7fc3d4159a10 legacy=0x7fc3d406ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d2ffd700 1 -- 172.21.15.16:0/333304153 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d406b6e0 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d2ffd700 1 --2- 172.21.15.16:0/333304153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ca0c0 0x7fc3d40a98b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ccad0 tx=0x7fc3c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea425030a637c1f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d27fc700 1 -- 172.21.15.16:0/333304153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011630 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d27fc700 1 -- 172.21.15.16:0/333304153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80117b0 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.271+0000 7fc3d27fc700 1 -- 172.21.15.16:0/333304153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8011b70 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 -- 172.21.15.16:0/333304153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ca0c0 msgr2=0x7fc3d40a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 --2- 172.21.15.16:0/333304153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ca0c0 0x7fc3d40a98b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40ccad0 tx=0x7fc3c80091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 -- 172.21.15.16:0/333304153 shutdown_connections 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 --2- 172.21.15.16:0/333304153 >> v2:172.21.15.177:3300/0 conn(0x7fc3d40c7370 0x7fc3d40c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 --2- 172.21.15.16:0/333304153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40ca0c0 0x7fc3d40a98b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 -- 172.21.15.16:0/333304153 >> 172.21.15.16:0/333304153 conn(0x7fc3d40cd690 msgr2=0x7fc3d40c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 -- 172.21.15.16:0/333304153 shutdown_connections 2023-10-06T11:42:49.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.272+0000 7fc3da15c700 1 -- 172.21.15.16:0/333304153 wait complete. 2023-10-06T11:42:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.273+0000 7fc3da15c700 1 Processor -- start 2023-10-06T11:42:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.273+0000 7fc3da15c700 1 -- start start 2023-10-06T11:42:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 0x7fc3d40abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3d40ca0c0 0x7fc3d415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40ad030 con 0x7fc3d4159a10 2023-10-06T11:42:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d415d2a0 con 0x7fc3d40c7370 2023-10-06T11:42:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40ac420 con 0x7fc3d40ca0c0 2023-10-06T11:42:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47542/0 (socket says 172.21.15.16:47542) 2023-10-06T11:42:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 -- 172.21.15.16:0/201087080 learned_addr learned my addr 172.21.15.16:0/201087080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 0x7fc3d40abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d2ffd700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3d40ca0c0 0x7fc3d415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 -- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3d40ca0c0 msgr2=0x7fc3d415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3d40ca0c0 0x7fc3d415cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 -- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 msgr2=0x7fc3d40abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 0x7fc3d40abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 -- 172.21.15.16:0/201087080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009420 con 0x7fc3d4159a10 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 0x7fc3d40abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3d3fff700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009df0 tx=0x7fc3cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc00cb50 con 0x7fc3d4159a10 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc00f040 con 0x7fc3d4159a10 2023-10-06T11:42:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc014480 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d40ac700 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.274+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d40accc0 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.275+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d404c380 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.276+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3cc014830 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.276+0000 7fc3bb7fe700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 0x7fc3c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.276+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc3cc085010 con 0x7fc3d4159a10 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.276+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 0x7fc3c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.277+0000 7fc3d37fe700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 0x7fc3c00693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40058d0 tx=0x7fc3c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.280+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3cc052120 con 0x7fc3d4159a10 2023-10-06T11:42:49.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.474+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d40ad170 con 0x7fc3d4159a10 2023-10-06T11:42:49.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.475+0000 7fc3bb7fe700 1 -- 172.21.15.16:0/201087080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fc3cc055740 con 0x7fc3d4159a10 2023-10-06T11:42:49.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:49.477 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:46.714204+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:42:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 msgr2=0x7fc3c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 0x7fc3c00693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40058d0 tx=0x7fc3c4009040 comp rx=0 tx=0).stop 2023-10-06T11:42:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 msgr2=0x7fc3d4158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009df0 tx=0x7fc3cc00b040 comp rx=0 tx=0).stop 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 shutdown_connections 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4159a10 0x7fc3d4158f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc3c0066f30 0x7fc3c00693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3d40c7370 0x7fc3d40abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 --2- 172.21.15.16:0/201087080 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3d40ca0c0 0x7fc3d415cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 >> 172.21.15.16:0/201087080 conn(0x7fc3d40cd690 msgr2=0x7fc3d40b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 shutdown_connections 2023-10-06T11:42:49.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.477+0000 7fc3da15c700 1 -- 172.21.15.16:0/201087080 wait complete. 2023-10-06T11:42:49.492 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T11:42:49.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.609+0000 7f321d7a0700 1 Processor -- start 2023-10-06T11:42:49.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 -- start start 2023-10-06T11:42:49.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f321815cc20 0x7f32180cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32180cf450 0x7f32180c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32180cdb10 con 0x7f32180cd6a0 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32180b3110 con 0x7f32180cf450 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f321d7a0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32180ccb20 con 0x7f321815cc20 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f32177fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32180cf450 0x7f32180c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f32177fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32180cf450 0x7f32180c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47546/0 (socket says 172.21.15.16:47546) 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.610+0000 7f32177fe700 1 -- 172.21.15.16:0/2607466605 learned_addr learned my addr 172.21.15.16:0/2607466605 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f3216ffd700 1 --2- 172.21.15.16:0/2607466605 >> v2:172.21.15.177:3300/0 conn(0x7f321815cc20 0x7f32180cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f32177fe700 1 -- 172.21.15.16:0/2607466605 >> v2:172.21.15.177:3300/0 conn(0x7f321815cc20 msgr2=0x7f32180cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f32177fe700 1 --2- 172.21.15.16:0/2607466605 >> v2:172.21.15.177:3300/0 conn(0x7f321815cc20 0x7f32180cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f32177fe700 1 -- 172.21.15.16:0/2607466605 >> v1:172.21.15.177:6789/0 conn(0x7f32180cd6a0 legacy=0x7f32180ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f32177fe700 1 -- 172.21.15.16:0/2607466605 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32180cf880 con 0x7f32180cf450 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f32177fe700 1 --2- 172.21.15.16:0/2607466605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cf450 0x7f32180c5590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3218089130 tx=0x7f3210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1dd962c4e0ee60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f3215ffb700 1 -- 172.21.15.16:0/2607466605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000b820 con 0x7f32180cf450 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f3215ffb700 1 -- 172.21.15.16:0/2607466605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000b980 con 0x7f32180cf450 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f3215ffb700 1 -- 172.21.15.16:0/2607466605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000bd40 con 0x7f32180cf450 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f321d7a0700 1 -- 172.21.15.16:0/2607466605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cf450 msgr2=0x7f32180c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.611+0000 7f321d7a0700 1 --2- 172.21.15.16:0/2607466605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cf450 0x7f32180c5590 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3218089130 tx=0x7f3210009250 comp rx=0 tx=0).stop 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 -- 172.21.15.16:0/2607466605 shutdown_connections 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 --2- 172.21.15.16:0/2607466605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cf450 0x7f32180c5590 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 --2- 172.21.15.16:0/2607466605 >> v2:172.21.15.177:3300/0 conn(0x7f321815cc20 0x7f32180cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 -- 172.21.15.16:0/2607466605 >> 172.21.15.16:0/2607466605 conn(0x7f3218099b40 msgr2=0x7f321806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 -- 172.21.15.16:0/2607466605 shutdown_connections 2023-10-06T11:42:49.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.612+0000 7f321d7a0700 1 -- 172.21.15.16:0/2607466605 wait complete. 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.613+0000 7f321d7a0700 1 Processor -- start 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.613+0000 7f321d7a0700 1 -- start start 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32180cf450 0x7f3218157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f321815cc20 0x7f32180c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32180c8e70 con 0x7f32180cd6a0 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32180c8260 con 0x7f32180cf450 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32180c8540 con 0x7f321815cc20 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47562/0 (socket says 172.21.15.16:47562) 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 -- 172.21.15.16:0/4116722882 learned_addr learned my addr 172.21.15.16:0/4116722882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 -- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f321815cc20 msgr2=0x7f32180c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f321815cc20 0x7f32180c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 -- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32180cf450 msgr2=0x7f3218157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f32167fc700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32180cf450 0x7f3218157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32180cf450 0x7f3218157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 -- 172.21.15.16:0/4116722882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100094f0 con 0x7f32180cd6a0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f3216ffd700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f320800e190 tx=0x7f3208009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320800ee30 con 0x7f32180cd6a0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208014420 con 0x7f32180cd6a0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32180c8820 con 0x7f32180cd6a0 2023-10-06T11:42:49.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32080147e0 con 0x7f32180cd6a0 2023-10-06T11:42:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.614+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321809e020 con 0x7f32180cd6a0 2023-10-06T11:42:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.615+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32180ce000 con 0x7f32180cd6a0 2023-10-06T11:42:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.616+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3208014b90 con 0x7f32180cd6a0 2023-10-06T11:42:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.616+0000 7f320effd700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 0x7f31f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.616+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3208084ec0 con 0x7f32180cd6a0 2023-10-06T11:42:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.616+0000 7f32167fc700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 0x7f31f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.617+0000 7f32167fc700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 0x7f31f80693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f320000a8a0 tx=0x7f3200008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.621+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3208051fd0 con 0x7f32180cd6a0 2023-10-06T11:42:49.699 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-10-06T11:42:49.697+0000 7f7778b91700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T11:42:49.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.809+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32180c8fb0 con 0x7f31f8066f30 2023-10-06T11:42:49.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.818+0000 7f320effd700 1 -- 172.21.15.16:0/4116722882 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276328 (secure 0 0 0) 0x7f320000c410 con 0x7f31f8066f30 2023-10-06T11:42:49.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 msgr2=0x7f31f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 0x7f31f80693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f320000a8a0 tx=0x7f3200008040 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 msgr2=0x7f32180c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f320800e190 tx=0x7f3208009920 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 shutdown_connections 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32180cd6a0 0x7f32180c9300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f31f8066f30 0x7f31f80693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f321815cc20 0x7f32180c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 --2- 172.21.15.16:0/4116722882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32180cf450 0x7f3218157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 >> 172.21.15.16:0/4116722882 conn(0x7f3218099b40 msgr2=0x7f32180c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 shutdown_connections 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.821+0000 7f321d7a0700 1 -- 172.21.15.16:0/4116722882 wait complete. 2023-10-06T11:42:49.826 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:49.835 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2023-10-06T11:42:48.072204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386004,"kb_used_data":3604,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588716,"statfs":{"total":773094113280,"available":770650845184,"internally_reserved":0,"allocated":3690496,"data_stored":2147110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.363250"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.716123+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:41.588797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712829+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:35.927136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713935+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:51.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712686+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:13.805975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.711395+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.008744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.714152+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:10.895046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713827+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:38:22.720411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712544+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:41:49.847225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713659+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:24.552817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.711424+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:10.895046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.711334+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:55:26.463987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713819+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:06.678801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.714090+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:59.090433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713967+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:55:26.463987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.716638+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:06:33.028929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713652+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:55:26.463987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712122+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:27.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712626+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:53.621481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.710573+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:10:16.695904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.714455+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:10.895046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.710675+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:07.662704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.710641+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:46:12.328038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713479+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:51.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.710586+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:21:38.671305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712549+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:13.673284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:38.715029+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:42:38.715029+0000","last_peered":"2023-10-06T11:42:38.715029+0000","last_clean":"2023-10-06T11:42:38.715029+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:42:38.715029+0000","last_undegraded":"2023-10-06T11:42:38.715029+0000","last_fullsized":"2023-10-06T11:42:38.715029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.713250+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:34.605521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:39.063785+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:42:39.063785+0000","last_peered":"2023-10-06T11:42:39.063785+0000","last_clean":"2023-10-06T11:42:39.063785+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:42:39.063785+0000","last_undegraded":"2023-10-06T11:42:39.063785+0000","last_fullsized":"2023-10-06T11:42:39.063785+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.711378+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:50:51.214737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:39.063543+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:42:39.063543+0000","last_peered":"2023-10-06T11:42:39.063543+0000","last_clean":"2023-10-06T11:42:39.063543+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:42:39.063543+0000","last_undegraded":"2023-10-06T11:42:39.063543+0000","last_fullsized":"2023-10-06T11:42:39.063543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:38.962512+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:42:38.962512+0000","last_peered":"2023-10-06T11:42:38.962512+0000","last_clean":"2023-10-06T11:42:38.962512+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:42:38.962512+0000","last_undegraded":"2023-10-06T11:42:38.962512+0000","last_fullsized":"2023-10-06T11:42:38.962512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.710590+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:20.976867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:38.962511+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:42:38.962511+0000","last_peered":"2023-10-06T11:42:38.962511+0000","last_clean":"2023-10-06T11:42:38.962511+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:42:38.962511+0000","last_undegraded":"2023-10-06T11:42:38.962511+0000","last_fullsized":"2023-10-06T11:42:38.962511+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.714670+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:27.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:38.962489+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:42:38.962489+0000","last_peered":"2023-10-06T11:42:38.962489+0000","last_clean":"2023-10-06T11:42:38.962489+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:42:38.962489+0000","last_undegraded":"2023-10-06T11:42:38.962489+0000","last_fullsized":"2023-10-06T11:42:38.962489+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712776+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:27.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:38.714480+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:42:38.714480+0000","last_peered":"2023-10-06T11:42:38.714480+0000","last_clean":"2023-10-06T11:42:38.714480+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:42:38.714480+0000","last_undegraded":"2023-10-06T11:42:38.714480+0000","last_fullsized":"2023-10-06T11:42:38.714480+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.716133+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:09.499866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:39.063825+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:42:39.063825+0000","last_peered":"2023-10-06T11:42:39.063825+0000","last_clean":"2023-10-06T11:42:39.063825+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:42:39.063825+0000","last_undegraded":"2023-10-06T11:42:39.063825+0000","last_fullsized":"2023-10-06T11:42:39.063825+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T11:42:44.708503+0000","last_change":"2023-10-06T11:42:44.708503+0000","last_active":"2023-10-06T11:42:44.708503+0000","last_peered":"2023-10-06T11:42:44.708503+0000","last_clean":"2023-10-06T11:42:44.708503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:44.708503+0000","last_undegraded":"2023-10-06T11:42:44.708503+0000","last_fullsized":"2023-10-06T11:42:44.708503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T11:42:43.703953+0000","last_change":"2023-10-06T11:42:43.712828+0000","last_active":"2023-10-06T11:42:43.703953+0000","last_peered":"2023-10-06T11:42:43.703953+0000","last_clean":"2023-10-06T11:42:43.703953+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T11:42:43.703953+0000","last_undegraded":"2023-10-06T11:42:43.703953+0000","last_fullsized":"2023-10-06T11:42:43.703953+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.008744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-10-06T11:42:39.063820+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:42:39.063820+0000","last_peered":"2023-10-06T11:42:39.063820+0000","last_clean":"2023-10-06T11:42:39.063820+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:42:39.063820+0000","last_undegraded":"2023-10-06T11:42:39.063820+0000","last_fullsized":"2023-10-06T11:42:39.063820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073668,"statfs":{"total":96636764160,"available":96331436032,"internally_reserved":0,"allocated":905216,"data_stored":709992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":124804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302268,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069572,"statfs":{"total":96636764160,"available":96327241728,"internally_reserved":0,"allocated":905216,"data_stored":709992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297588,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074252,"statfs":{"total":96636764160,"available":96332034048,"internally_reserved":0,"allocated":307200,"data_stored":118646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":119624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":124804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":119624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":311296,"data_stored":119624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:49.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:42:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.993+0000 7fcfee64f700 1 Processor -- start 2023-10-06T11:42:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.993+0000 7fcfee64f700 1 -- start start 2023-10-06T11:42:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfee64f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfe80af8d0 0x7fcfe815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:49.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfee64f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe815da60 con 0x7fcfe80cc890 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfee64f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe8159dd0 con 0x7fcfe806c410 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfee64f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe815e530 con 0x7fcfe80af8d0 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfe80af8d0 0x7fcfe815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcfe80cc890 0x7fcfe80675c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36284/0 (socket says 172.21.15.16:36284) 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe7fff700 1 -- 172.21.15.16:0/2819983130 learned_addr learned my addr 172.21.15.16:0/2819983130 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe77fe700 1 -- 172.21.15.16:0/2819983130 >> v1:172.21.15.177:6789/0 conn(0x7fcfe806c410 legacy=0x7fcfe80b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe77fe700 1 -- 172.21.15.16:0/2819983130 >> v1:172.21.15.16:6789/0 conn(0x7fcfe80cc890 legacy=0x7fcfe80675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe77fe700 1 -- 172.21.15.16:0/2819983130 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe815dc90 con 0x7fcfe80af8d0 2023-10-06T11:42:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe6ffd700 1 -- 172.21.15.16:0/2819983130 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597062931 0 0) 0x7fcfe815da60 con 0x7fcfe80cc890 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.994+0000 7fcfe77fe700 1 --2- 172.21.15.16:0/2819983130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815d520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009910 tx=0x7fcfd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df941b456ae14ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfe6ffd700 1 -- 172.21.15.16:0/2819983130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800b7e0 con 0x7fcfe80af8d0 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfe6ffd700 1 -- 172.21.15.16:0/2819983130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd800b940 con 0x7fcfe80af8d0 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfe6ffd700 1 -- 172.21.15.16:0/2819983130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800bce0 con 0x7fcfe80af8d0 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 -- 172.21.15.16:0/2819983130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80af8d0 msgr2=0x7fcfe815d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 --2- 172.21.15.16:0/2819983130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815d520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009910 tx=0x7fcfd80091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 -- 172.21.15.16:0/2819983130 shutdown_connections 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 --2- 172.21.15.16:0/2819983130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815d520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 -- 172.21.15.16:0/2819983130 >> 172.21.15.16:0/2819983130 conn(0x7fcfe806dfa0 msgr2=0x7fcfe80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 -- 172.21.15.16:0/2819983130 shutdown_connections 2023-10-06T11:42:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.995+0000 7fcfee64f700 1 -- 172.21.15.16:0/2819983130 wait complete. 2023-10-06T11:42:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.996+0000 7fcfee64f700 1 Processor -- start 2023-10-06T11:42:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 -- start start 2023-10-06T11:42:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfe80cc890 0x7fcfe816ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe80ac770 con 0x7fcfe806c410 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8155000 con 0x7fcfe80af8d0 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfee64f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe81552e0 con 0x7fcfe80cc890 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47568/0 (socket says 172.21.15.16:47568) 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfe7fff700 1 -- 172.21.15.16:0/4168354861 learned_addr learned my addr 172.21.15.16:0/4168354861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.997+0000 7fcfe77fe700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfecbec700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfe80cc890 0x7fcfe816ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 -- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfe80cc890 msgr2=0x7fcfe816ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfe80cc890 0x7fcfe816ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 -- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 msgr2=0x7fcfe815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 -- 172.21.15.16:0/4168354861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8009420 con 0x7fcfe806c410 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe77fe700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfe7fff700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c5b0 tx=0x7fcfd0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0005b30 con 0x7fcfe806c410 2023-10-06T11:42:50.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0005c90 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0007300 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe81555c0 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.998+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe80ac480 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.999+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe804c380 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:49.999+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfd001b3b0 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.000+0000 7fcfdffff700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.000+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcfd0083bc0 con 0x7fcfe806c410 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.000+0000 7fcfe77fe700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.001+0000 7fcfe77fe700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 0x7fcfc80693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80093f0 tx=0x7fcfd8016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.004+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfd0050cd0 con 0x7fcfe806c410 2023-10-06T11:42:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.192+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfe806d1d0 con 0x7fcfc8066f30 2023-10-06T11:42:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.200+0000 7fcfdffff700 1 -- 172.21.15.16:0/4168354861 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292593 (secure 0 0 0) 0x7fcfe806d1d0 con 0x7fcfc8066f30 2023-10-06T11:42:50.206 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 msgr2=0x7fcfc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 0x7fcfc80693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80093f0 tx=0x7fcfd8016000 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 msgr2=0x7fcfe80abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c5b0 tx=0x7fcfd0001070 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 shutdown_connections 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe806c410 0x7fcfe80abf40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcfc8066f30 0x7fcfc80693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfe80cc890 0x7fcfe816ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 --2- 172.21.15.16:0/4168354861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfe80af8d0 0x7fcfe815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 >> 172.21.15.16:0/4168354861 conn(0x7fcfe806dfa0 msgr2=0x7fcfe80ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:50.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 shutdown_connections 2023-10-06T11:42:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.204+0000 7fcfee64f700 1 -- 172.21.15.16:0/4168354861 wait complete. 2023-10-06T11:42:50.210 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:42:50.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2023-10-06T11:42:50.072649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386428,"kb_used_data":4028,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588292,"statfs":{"total":773094113280,"available":770650411008,"internally_reserved":0,"allocated":4124672,"data_stored":2351654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001417"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749017+0000","last_change":"2023-10-06T11:42:45.741368+0000","last_active":"2023-10-06T11:42:45.749017+0000","last_peered":"2023-10-06T11:42:45.749017+0000","last_clean":"2023-10-06T11:42:45.749017+0000","last_became_active":"2023-10-06T11:42:45.740900+0000","last_became_peered":"2023-10-06T11:42:45.740900+0000","last_unstale":"2023-10-06T11:42:45.749017+0000","last_undegraded":"2023-10-06T11:42:45.749017+0000","last_fullsized":"2023-10-06T11:42:45.749017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:20.769800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749109+0000","last_change":"2023-10-06T11:42:45.741696+0000","last_active":"2023-10-06T11:42:45.749109+0000","last_peered":"2023-10-06T11:42:45.749109+0000","last_clean":"2023-10-06T11:42:45.749109+0000","last_became_active":"2023-10-06T11:42:45.741179+0000","last_became_peered":"2023-10-06T11:42:45.741179+0000","last_unstale":"2023-10-06T11:42:45.749109+0000","last_undegraded":"2023-10-06T11:42:45.749109+0000","last_fullsized":"2023-10-06T11:42:45.749109+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:02.157089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.734045+0000","last_change":"2023-10-06T11:42:45.729984+0000","last_active":"2023-10-06T11:42:45.734045+0000","last_peered":"2023-10-06T11:42:45.734045+0000","last_clean":"2023-10-06T11:42:45.734045+0000","last_became_active":"2023-10-06T11:42:45.729695+0000","last_became_peered":"2023-10-06T11:42:45.729695+0000","last_unstale":"2023-10-06T11:42:45.734045+0000","last_undegraded":"2023-10-06T11:42:45.734045+0000","last_fullsized":"2023-10-06T11:42:45.734045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:22:02.515634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.745762+0000","last_change":"2023-10-06T11:42:45.743265+0000","last_active":"2023-10-06T11:42:45.745762+0000","last_peered":"2023-10-06T11:42:45.745762+0000","last_clean":"2023-10-06T11:42:45.745762+0000","last_became_active":"2023-10-06T11:42:45.742637+0000","last_became_peered":"2023-10-06T11:42:45.742637+0000","last_unstale":"2023-10-06T11:42:45.745762+0000","last_undegraded":"2023-10-06T11:42:45.745762+0000","last_fullsized":"2023-10-06T11:42:45.745762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:36.931685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.747510+0000","last_change":"2023-10-06T11:42:45.742965+0000","last_active":"2023-10-06T11:42:45.747510+0000","last_peered":"2023-10-06T11:42:45.747510+0000","last_clean":"2023-10-06T11:42:45.747510+0000","last_became_active":"2023-10-06T11:42:45.742704+0000","last_became_peered":"2023-10-06T11:42:45.742704+0000","last_unstale":"2023-10-06T11:42:45.747510+0000","last_undegraded":"2023-10-06T11:42:45.747510+0000","last_fullsized":"2023-10-06T11:42:45.747510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:23.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749837+0000","last_change":"2023-10-06T11:42:45.748693+0000","last_active":"2023-10-06T11:42:45.749837+0000","last_peered":"2023-10-06T11:42:45.749837+0000","last_clean":"2023-10-06T11:42:45.749837+0000","last_became_active":"2023-10-06T11:42:45.748398+0000","last_became_peered":"2023-10-06T11:42:45.748398+0000","last_unstale":"2023-10-06T11:42:45.749837+0000","last_undegraded":"2023-10-06T11:42:45.749837+0000","last_fullsized":"2023-10-06T11:42:45.749837+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:14.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743947+0000","last_change":"2023-10-06T11:42:45.740639+0000","last_active":"2023-10-06T11:42:45.743947+0000","last_peered":"2023-10-06T11:42:45.743947+0000","last_clean":"2023-10-06T11:42:45.743947+0000","last_became_active":"2023-10-06T11:42:45.740335+0000","last_became_peered":"2023-10-06T11:42:45.740335+0000","last_unstale":"2023-10-06T11:42:45.743947+0000","last_undegraded":"2023-10-06T11:42:45.743947+0000","last_fullsized":"2023-10-06T11:42:45.743947+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.258996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749179+0000","last_change":"2023-10-06T11:42:45.741320+0000","last_active":"2023-10-06T11:42:45.749179+0000","last_peered":"2023-10-06T11:42:45.749179+0000","last_clean":"2023-10-06T11:42:45.749179+0000","last_became_active":"2023-10-06T11:42:45.740728+0000","last_became_peered":"2023-10-06T11:42:45.740728+0000","last_unstale":"2023-10-06T11:42:45.749179+0000","last_undegraded":"2023-10-06T11:42:45.749179+0000","last_fullsized":"2023-10-06T11:42:45.749179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:49.642325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.733946+0000","last_change":"2023-10-06T11:42:45.729871+0000","last_active":"2023-10-06T11:42:45.733946+0000","last_peered":"2023-10-06T11:42:45.733946+0000","last_clean":"2023-10-06T11:42:45.733946+0000","last_became_active":"2023-10-06T11:42:45.728462+0000","last_became_peered":"2023-10-06T11:42:45.728462+0000","last_unstale":"2023-10-06T11:42:45.733946+0000","last_undegraded":"2023-10-06T11:42:45.733946+0000","last_fullsized":"2023-10-06T11:42:45.733946+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:54.626031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.729099+0000","last_change":"2023-10-06T11:42:45.726120+0000","last_active":"2023-10-06T11:42:45.729099+0000","last_peered":"2023-10-06T11:42:45.729099+0000","last_clean":"2023-10-06T11:42:45.729099+0000","last_became_active":"2023-10-06T11:42:45.725772+0000","last_became_peered":"2023-10-06T11:42:45.725772+0000","last_unstale":"2023-10-06T11:42:45.729099+0000","last_undegraded":"2023-10-06T11:42:45.729099+0000","last_fullsized":"2023-10-06T11:42:45.729099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:51.840590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.732421+0000","last_change":"2023-10-06T11:42:45.727264+0000","last_active":"2023-10-06T11:42:45.732421+0000","last_peered":"2023-10-06T11:42:45.732421+0000","last_clean":"2023-10-06T11:42:45.732421+0000","last_became_active":"2023-10-06T11:42:45.726895+0000","last_became_peered":"2023-10-06T11:42:45.726895+0000","last_unstale":"2023-10-06T11:42:45.732421+0000","last_undegraded":"2023-10-06T11:42:45.732421+0000","last_fullsized":"2023-10-06T11:42:45.732421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:52.510854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.733845+0000","last_change":"2023-10-06T11:42:45.729938+0000","last_active":"2023-10-06T11:42:45.733845+0000","last_peered":"2023-10-06T11:42:45.733845+0000","last_clean":"2023-10-06T11:42:45.733845+0000","last_became_active":"2023-10-06T11:42:45.729471+0000","last_became_peered":"2023-10-06T11:42:45.729471+0000","last_unstale":"2023-10-06T11:42:45.733845+0000","last_undegraded":"2023-10-06T11:42:45.733845+0000","last_fullsized":"2023-10-06T11:42:45.733845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:14.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.732417+0000","last_change":"2023-10-06T11:42:45.726319+0000","last_active":"2023-10-06T11:42:45.732417+0000","last_peered":"2023-10-06T11:42:45.732417+0000","last_clean":"2023-10-06T11:42:45.732417+0000","last_became_active":"2023-10-06T11:42:45.725530+0000","last_became_peered":"2023-10-06T11:42:45.725530+0000","last_unstale":"2023-10-06T11:42:45.732417+0000","last_undegraded":"2023-10-06T11:42:45.732417+0000","last_fullsized":"2023-10-06T11:42:45.732417+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:40.606457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741720+0000","last_change":"2023-10-06T11:42:45.736037+0000","last_active":"2023-10-06T11:42:45.741720+0000","last_peered":"2023-10-06T11:42:45.741720+0000","last_clean":"2023-10-06T11:42:45.741720+0000","last_became_active":"2023-10-06T11:42:45.735784+0000","last_became_peered":"2023-10-06T11:42:45.735784+0000","last_unstale":"2023-10-06T11:42:45.741720+0000","last_undegraded":"2023-10-06T11:42:45.741720+0000","last_fullsized":"2023-10-06T11:42:45.741720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:43:23.562691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.730525+0000","last_change":"2023-10-06T11:42:45.725444+0000","last_active":"2023-10-06T11:42:45.730525+0000","last_peered":"2023-10-06T11:42:45.730525+0000","last_clean":"2023-10-06T11:42:45.730525+0000","last_became_active":"2023-10-06T11:42:45.725139+0000","last_became_peered":"2023-10-06T11:42:45.725139+0000","last_unstale":"2023-10-06T11:42:45.730525+0000","last_undegraded":"2023-10-06T11:42:45.730525+0000","last_fullsized":"2023-10-06T11:42:45.730525+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:02.157089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.733987+0000","last_change":"2023-10-06T11:42:45.729827+0000","last_active":"2023-10-06T11:42:45.733987+0000","last_peered":"2023-10-06T11:42:45.733987+0000","last_clean":"2023-10-06T11:42:45.733987+0000","last_became_active":"2023-10-06T11:42:45.729562+0000","last_became_peered":"2023-10-06T11:42:45.729562+0000","last_unstale":"2023-10-06T11:42:45.733987+0000","last_undegraded":"2023-10-06T11:42:45.733987+0000","last_fullsized":"2023-10-06T11:42:45.733987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:40.606457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.733995+0000","last_change":"2023-10-06T11:42:45.731495+0000","last_active":"2023-10-06T11:42:45.733995+0000","last_peered":"2023-10-06T11:42:45.733995+0000","last_clean":"2023-10-06T11:42:45.733995+0000","last_became_active":"2023-10-06T11:42:45.731184+0000","last_became_peered":"2023-10-06T11:42:45.731184+0000","last_unstale":"2023-10-06T11:42:45.733995+0000","last_undegraded":"2023-10-06T11:42:45.733995+0000","last_fullsized":"2023-10-06T11:42:45.733995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:21:14.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743324+0000","last_change":"2023-10-06T11:42:45.738745+0000","last_active":"2023-10-06T11:42:45.743324+0000","last_peered":"2023-10-06T11:42:45.743324+0000","last_clean":"2023-10-06T11:42:45.743324+0000","last_became_active":"2023-10-06T11:42:45.738551+0000","last_became_peered":"2023-10-06T11:42:45.738551+0000","last_unstale":"2023-10-06T11:42:45.743324+0000","last_undegraded":"2023-10-06T11:42:45.743324+0000","last_fullsized":"2023-10-06T11:42:45.743324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:07.683350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.744679+0000","last_change":"2023-10-06T11:42:45.737076+0000","last_active":"2023-10-06T11:42:45.744679+0000","last_peered":"2023-10-06T11:42:45.744679+0000","last_clean":"2023-10-06T11:42:45.744679+0000","last_became_active":"2023-10-06T11:42:45.736779+0000","last_became_peered":"2023-10-06T11:42:45.736779+0000","last_unstale":"2023-10-06T11:42:45.744679+0000","last_undegraded":"2023-10-06T11:42:45.744679+0000","last_fullsized":"2023-10-06T11:42:45.744679+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:54:11.899595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743330+0000","last_change":"2023-10-06T11:42:45.738816+0000","last_active":"2023-10-06T11:42:45.743330+0000","last_peered":"2023-10-06T11:42:45.743330+0000","last_clean":"2023-10-06T11:42:45.743330+0000","last_became_active":"2023-10-06T11:42:45.738541+0000","last_became_peered":"2023-10-06T11:42:45.738541+0000","last_unstale":"2023-10-06T11:42:45.743330+0000","last_undegraded":"2023-10-06T11:42:45.743330+0000","last_fullsized":"2023-10-06T11:42:45.743330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:35.610070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743916+0000","last_change":"2023-10-06T11:42:45.737418+0000","last_active":"2023-10-06T11:42:45.743916+0000","last_peered":"2023-10-06T11:42:45.743916+0000","last_clean":"2023-10-06T11:42:45.743916+0000","last_became_active":"2023-10-06T11:42:45.737084+0000","last_became_peered":"2023-10-06T11:42:45.737084+0000","last_unstale":"2023-10-06T11:42:45.743916+0000","last_undegraded":"2023-10-06T11:42:45.743916+0000","last_fullsized":"2023-10-06T11:42:45.743916+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:37:20.769800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742542+0000","last_change":"2023-10-06T11:42:45.740086+0000","last_active":"2023-10-06T11:42:45.742542+0000","last_peered":"2023-10-06T11:42:45.742542+0000","last_clean":"2023-10-06T11:42:45.742542+0000","last_became_active":"2023-10-06T11:42:45.739825+0000","last_became_peered":"2023-10-06T11:42:45.739825+0000","last_unstale":"2023-10-06T11:42:45.742542+0000","last_undegraded":"2023-10-06T11:42:45.742542+0000","last_fullsized":"2023-10-06T11:42:45.742542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:36:49.976138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742762+0000","last_change":"2023-10-06T11:42:45.736996+0000","last_active":"2023-10-06T11:42:45.742762+0000","last_peered":"2023-10-06T11:42:45.742762+0000","last_clean":"2023-10-06T11:42:45.742762+0000","last_became_active":"2023-10-06T11:42:45.736655+0000","last_became_peered":"2023-10-06T11:42:45.736655+0000","last_unstale":"2023-10-06T11:42:45.742762+0000","last_undegraded":"2023-10-06T11:42:45.742762+0000","last_fullsized":"2023-10-06T11:42:45.742762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:51.840590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.740101+0000","last_change":"2023-10-06T11:42:45.735950+0000","last_active":"2023-10-06T11:42:45.740101+0000","last_peered":"2023-10-06T11:42:45.740101+0000","last_clean":"2023-10-06T11:42:45.740101+0000","last_became_active":"2023-10-06T11:42:45.735667+0000","last_became_peered":"2023-10-06T11:42:45.735667+0000","last_unstale":"2023-10-06T11:42:45.740101+0000","last_undegraded":"2023-10-06T11:42:45.740101+0000","last_fullsized":"2023-10-06T11:42:45.740101+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:35.610070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.734291+0000","last_change":"2023-10-06T11:42:45.726536+0000","last_active":"2023-10-06T11:42:45.734291+0000","last_peered":"2023-10-06T11:42:45.734291+0000","last_clean":"2023-10-06T11:42:45.734291+0000","last_became_active":"2023-10-06T11:42:45.726303+0000","last_became_peered":"2023-10-06T11:42:45.726303+0000","last_unstale":"2023-10-06T11:42:45.734291+0000","last_undegraded":"2023-10-06T11:42:45.734291+0000","last_fullsized":"2023-10-06T11:42:45.734291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:25.557367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.746713+0000","last_change":"2023-10-06T11:42:45.743485+0000","last_active":"2023-10-06T11:42:45.746713+0000","last_peered":"2023-10-06T11:42:45.746713+0000","last_clean":"2023-10-06T11:42:45.746713+0000","last_became_active":"2023-10-06T11:42:45.743159+0000","last_became_peered":"2023-10-06T11:42:45.743159+0000","last_unstale":"2023-10-06T11:42:45.746713+0000","last_undegraded":"2023-10-06T11:42:45.746713+0000","last_fullsized":"2023-10-06T11:42:45.746713+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:14.677833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.735068+0000","last_change":"2023-10-06T11:42:45.729018+0000","last_active":"2023-10-06T11:42:45.735068+0000","last_peered":"2023-10-06T11:42:45.735068+0000","last_clean":"2023-10-06T11:42:45.735068+0000","last_became_active":"2023-10-06T11:42:45.728777+0000","last_became_peered":"2023-10-06T11:42:45.728777+0000","last_unstale":"2023-10-06T11:42:45.735068+0000","last_undegraded":"2023-10-06T11:42:45.735068+0000","last_fullsized":"2023-10-06T11:42:45.735068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:23.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.744017+0000","last_change":"2023-10-06T11:42:45.741307+0000","last_active":"2023-10-06T11:42:45.744017+0000","last_peered":"2023-10-06T11:42:45.744017+0000","last_clean":"2023-10-06T11:42:45.744017+0000","last_became_active":"2023-10-06T11:42:45.741041+0000","last_became_peered":"2023-10-06T11:42:45.741041+0000","last_unstale":"2023-10-06T11:42:45.744017+0000","last_undegraded":"2023-10-06T11:42:45.744017+0000","last_fullsized":"2023-10-06T11:42:45.744017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:49.642325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741632+0000","last_change":"2023-10-06T11:42:45.736423+0000","last_active":"2023-10-06T11:42:45.741632+0000","last_peered":"2023-10-06T11:42:45.741632+0000","last_clean":"2023-10-06T11:42:45.741632+0000","last_became_active":"2023-10-06T11:42:45.736168+0000","last_became_peered":"2023-10-06T11:42:45.736168+0000","last_unstale":"2023-10-06T11:42:45.741632+0000","last_undegraded":"2023-10-06T11:42:45.741632+0000","last_fullsized":"2023-10-06T11:42:45.741632+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:00.094983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743795+0000","last_change":"2023-10-06T11:42:45.741087+0000","last_active":"2023-10-06T11:42:45.743795+0000","last_peered":"2023-10-06T11:42:45.743795+0000","last_clean":"2023-10-06T11:42:45.743795+0000","last_became_active":"2023-10-06T11:42:45.740797+0000","last_became_peered":"2023-10-06T11:42:45.740797+0000","last_unstale":"2023-10-06T11:42:45.743795+0000","last_undegraded":"2023-10-06T11:42:45.743795+0000","last_fullsized":"2023-10-06T11:42:45.743795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:36.931685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742780+0000","last_change":"2023-10-06T11:42:45.737833+0000","last_active":"2023-10-06T11:42:45.742780+0000","last_peered":"2023-10-06T11:42:45.742780+0000","last_clean":"2023-10-06T11:42:45.742780+0000","last_became_active":"2023-10-06T11:42:45.737560+0000","last_became_peered":"2023-10-06T11:42:45.737560+0000","last_unstale":"2023-10-06T11:42:45.742780+0000","last_undegraded":"2023-10-06T11:42:45.742780+0000","last_fullsized":"2023-10-06T11:42:45.742780+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:32:33.819762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743925+0000","last_change":"2023-10-06T11:42:45.737578+0000","last_active":"2023-10-06T11:42:45.743925+0000","last_peered":"2023-10-06T11:42:45.743925+0000","last_clean":"2023-10-06T11:42:45.743925+0000","last_became_active":"2023-10-06T11:42:45.735794+0000","last_became_peered":"2023-10-06T11:42:45.735794+0000","last_unstale":"2023-10-06T11:42:45.743925+0000","last_undegraded":"2023-10-06T11:42:45.743925+0000","last_fullsized":"2023-10-06T11:42:45.743925+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:24:41.685903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.738187+0000","last_change":"2023-10-06T11:42:45.733295+0000","last_active":"2023-10-06T11:42:45.738187+0000","last_peered":"2023-10-06T11:42:45.738187+0000","last_clean":"2023-10-06T11:42:45.738187+0000","last_became_active":"2023-10-06T11:42:45.733050+0000","last_became_peered":"2023-10-06T11:42:45.733050+0000","last_unstale":"2023-10-06T11:42:45.738187+0000","last_undegraded":"2023-10-06T11:42:45.738187+0000","last_fullsized":"2023-10-06T11:42:45.738187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:52.510854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742542+0000","last_change":"2023-10-06T11:42:44.735523+0000","last_active":"2023-10-06T11:42:45.742542+0000","last_peered":"2023-10-06T11:42:45.742542+0000","last_clean":"2023-10-06T11:42:45.742542+0000","last_became_active":"2023-10-06T11:42:44.735293+0000","last_became_peered":"2023-10-06T11:42:44.735293+0000","last_unstale":"2023-10-06T11:42:45.742542+0000","last_undegraded":"2023-10-06T11:42:45.742542+0000","last_fullsized":"2023-10-06T11:42:45.742542+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:37:15.231524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743986+0000","last_change":"2023-10-06T11:42:45.735385+0000","last_active":"2023-10-06T11:42:45.743986+0000","last_peered":"2023-10-06T11:42:45.743986+0000","last_clean":"2023-10-06T11:42:45.743986+0000","last_became_active":"2023-10-06T11:42:45.732967+0000","last_became_peered":"2023-10-06T11:42:45.732967+0000","last_unstale":"2023-10-06T11:42:45.743986+0000","last_undegraded":"2023-10-06T11:42:45.743986+0000","last_fullsized":"2023-10-06T11:42:45.743986+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:34:54.626031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.738904+0000","last_change":"2023-10-06T11:42:44.728152+0000","last_active":"2023-10-06T11:42:45.738904+0000","last_peered":"2023-10-06T11:42:45.738904+0000","last_clean":"2023-10-06T11:42:45.738904+0000","last_became_active":"2023-10-06T11:42:44.727869+0000","last_became_peered":"2023-10-06T11:42:44.727869+0000","last_unstale":"2023-10-06T11:42:45.738904+0000","last_undegraded":"2023-10-06T11:42:45.738904+0000","last_fullsized":"2023-10-06T11:42:45.738904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:34.605521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741692+0000","last_change":"2023-10-06T11:42:45.733394+0000","last_active":"2023-10-06T11:42:45.741692+0000","last_peered":"2023-10-06T11:42:45.741692+0000","last_clean":"2023-10-06T11:42:45.741692+0000","last_became_active":"2023-10-06T11:42:45.732794+0000","last_became_peered":"2023-10-06T11:42:45.732794+0000","last_unstale":"2023-10-06T11:42:45.741692+0000","last_undegraded":"2023-10-06T11:42:45.741692+0000","last_fullsized":"2023-10-06T11:42:45.741692+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:04:05.362618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.746836+0000","last_change":"2023-10-06T11:42:44.733590+0000","last_active":"2023-10-06T11:42:45.746836+0000","last_peered":"2023-10-06T11:42:45.746836+0000","last_clean":"2023-10-06T11:42:45.746836+0000","last_became_active":"2023-10-06T11:42:44.732621+0000","last_became_peered":"2023-10-06T11:42:44.732621+0000","last_unstale":"2023-10-06T11:42:45.746836+0000","last_undegraded":"2023-10-06T11:42:45.746836+0000","last_fullsized":"2023-10-06T11:42:45.746836+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:34.605521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.748992+0000","last_change":"2023-10-06T11:42:45.741173+0000","last_active":"2023-10-06T11:42:45.748992+0000","last_peered":"2023-10-06T11:42:45.748992+0000","last_clean":"2023-10-06T11:42:45.748992+0000","last_became_active":"2023-10-06T11:42:45.740481+0000","last_became_peered":"2023-10-06T11:42:45.740481+0000","last_unstale":"2023-10-06T11:42:45.748992+0000","last_undegraded":"2023-10-06T11:42:45.748992+0000","last_fullsized":"2023-10-06T11:42:45.748992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:20:08.364640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.747314+0000","last_change":"2023-10-06T11:42:44.730710+0000","last_active":"2023-10-06T11:42:45.747314+0000","last_peered":"2023-10-06T11:42:45.747314+0000","last_clean":"2023-10-06T11:42:45.747314+0000","last_became_active":"2023-10-06T11:42:44.730337+0000","last_became_peered":"2023-10-06T11:42:44.730337+0000","last_unstale":"2023-10-06T11:42:45.747314+0000","last_undegraded":"2023-10-06T11:42:45.747314+0000","last_fullsized":"2023-10-06T11:42:45.747314+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:50.836040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.739236+0000","last_change":"2023-10-06T11:42:45.736800+0000","last_active":"2023-10-06T11:42:45.739236+0000","last_peered":"2023-10-06T11:42:45.739236+0000","last_clean":"2023-10-06T11:42:45.739236+0000","last_became_active":"2023-10-06T11:42:45.736545+0000","last_became_peered":"2023-10-06T11:42:45.736545+0000","last_unstale":"2023-10-06T11:42:45.739236+0000","last_undegraded":"2023-10-06T11:42:45.739236+0000","last_fullsized":"2023-10-06T11:42:45.739236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:52:14.677833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.732704+0000","last_change":"2023-10-06T11:42:44.730838+0000","last_active":"2023-10-06T11:42:45.732704+0000","last_peered":"2023-10-06T11:42:45.732704+0000","last_clean":"2023-10-06T11:42:45.732704+0000","last_became_active":"2023-10-06T11:42:44.730573+0000","last_became_peered":"2023-10-06T11:42:44.730573+0000","last_unstale":"2023-10-06T11:42:45.732704+0000","last_undegraded":"2023-10-06T11:42:45.732704+0000","last_fullsized":"2023-10-06T11:42:45.732704+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:06.678801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.730545+0000","last_change":"2023-10-06T11:42:45.725551+0000","last_active":"2023-10-06T11:42:45.730545+0000","last_peered":"2023-10-06T11:42:45.730545+0000","last_clean":"2023-10-06T11:42:45.730545+0000","last_became_active":"2023-10-06T11:42:45.725258+0000","last_became_peered":"2023-10-06T11:42:45.725258+0000","last_unstale":"2023-10-06T11:42:45.730545+0000","last_undegraded":"2023-10-06T11:42:45.730545+0000","last_fullsized":"2023-10-06T11:42:45.730545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:24.038405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741828+0000","last_change":"2023-10-06T11:42:44.733148+0000","last_active":"2023-10-06T11:42:45.741828+0000","last_peered":"2023-10-06T11:42:45.741828+0000","last_clean":"2023-10-06T11:42:45.741828+0000","last_became_active":"2023-10-06T11:42:44.732851+0000","last_became_peered":"2023-10-06T11:42:44.732851+0000","last_unstale":"2023-10-06T11:42:45.741828+0000","last_undegraded":"2023-10-06T11:42:45.741828+0000","last_fullsized":"2023-10-06T11:42:45.741828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:35.927136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741300+0000","last_change":"2023-10-06T11:42:45.728459+0000","last_active":"2023-10-06T11:42:45.741300+0000","last_peered":"2023-10-06T11:42:45.741300+0000","last_clean":"2023-10-06T11:42:45.741300+0000","last_became_active":"2023-10-06T11:42:45.727028+0000","last_became_peered":"2023-10-06T11:42:45.727028+0000","last_unstale":"2023-10-06T11:42:45.741300+0000","last_undegraded":"2023-10-06T11:42:45.741300+0000","last_fullsized":"2023-10-06T11:42:45.741300+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:59.074570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.728102+0000","last_change":"2023-10-06T11:42:44.728542+0000","last_active":"2023-10-06T11:42:45.728102+0000","last_peered":"2023-10-06T11:42:45.728102+0000","last_clean":"2023-10-06T11:42:45.728102+0000","last_became_active":"2023-10-06T11:42:44.728089+0000","last_became_peered":"2023-10-06T11:42:44.728089+0000","last_unstale":"2023-10-06T11:42:45.728102+0000","last_undegraded":"2023-10-06T11:42:45.728102+0000","last_fullsized":"2023-10-06T11:42:45.728102+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:27.671994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749407+0000","last_change":"2023-10-06T11:42:45.744891+0000","last_active":"2023-10-06T11:42:45.749407+0000","last_peered":"2023-10-06T11:42:45.749407+0000","last_clean":"2023-10-06T11:42:45.749407+0000","last_became_active":"2023-10-06T11:42:45.744391+0000","last_became_peered":"2023-10-06T11:42:45.744391+0000","last_unstale":"2023-10-06T11:42:45.749407+0000","last_undegraded":"2023-10-06T11:42:45.749407+0000","last_fullsized":"2023-10-06T11:42:45.749407+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:51.840590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.718743+0000","last_change":"2023-10-06T11:42:44.730825+0000","last_active":"2023-10-06T11:42:45.718743+0000","last_peered":"2023-10-06T11:42:45.718743+0000","last_clean":"2023-10-06T11:42:45.718743+0000","last_became_active":"2023-10-06T11:42:44.730568+0000","last_became_peered":"2023-10-06T11:42:44.730568+0000","last_unstale":"2023-10-06T11:42:45.718743+0000","last_undegraded":"2023-10-06T11:42:45.718743+0000","last_fullsized":"2023-10-06T11:42:45.718743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:51.506305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.745781+0000","last_change":"2023-10-06T11:42:45.743449+0000","last_active":"2023-10-06T11:42:45.745781+0000","last_peered":"2023-10-06T11:42:45.745781+0000","last_clean":"2023-10-06T11:42:45.745781+0000","last_became_active":"2023-10-06T11:42:45.742996+0000","last_became_peered":"2023-10-06T11:42:45.742996+0000","last_unstale":"2023-10-06T11:42:45.745781+0000","last_undegraded":"2023-10-06T11:42:45.745781+0000","last_fullsized":"2023-10-06T11:42:45.745781+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:44:28.676543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.727153+0000","last_change":"2023-10-06T11:42:44.731177+0000","last_active":"2023-10-06T11:42:45.727153+0000","last_peered":"2023-10-06T11:42:45.727153+0000","last_clean":"2023-10-06T11:42:45.727153+0000","last_became_active":"2023-10-06T11:42:44.730885+0000","last_became_peered":"2023-10-06T11:42:44.730885+0000","last_unstale":"2023-10-06T11:42:45.727153+0000","last_undegraded":"2023-10-06T11:42:45.727153+0000","last_fullsized":"2023-10-06T11:42:45.727153+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:09:12.992444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.737119+0000","last_change":"2023-10-06T11:42:45.733256+0000","last_active":"2023-10-06T11:42:45.737119+0000","last_peered":"2023-10-06T11:42:45.737119+0000","last_clean":"2023-10-06T11:42:45.737119+0000","last_became_active":"2023-10-06T11:42:45.732998+0000","last_became_peered":"2023-10-06T11:42:45.732998+0000","last_unstale":"2023-10-06T11:42:45.737119+0000","last_undegraded":"2023-10-06T11:42:45.737119+0000","last_fullsized":"2023-10-06T11:42:45.737119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:41.173527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.720023+0000","last_change":"2023-10-06T11:42:44.731199+0000","last_active":"2023-10-06T11:42:45.720023+0000","last_peered":"2023-10-06T11:42:45.720023+0000","last_clean":"2023-10-06T11:42:45.720023+0000","last_became_active":"2023-10-06T11:42:44.730981+0000","last_became_peered":"2023-10-06T11:42:44.730981+0000","last_unstale":"2023-10-06T11:42:45.720023+0000","last_undegraded":"2023-10-06T11:42:45.720023+0000","last_fullsized":"2023-10-06T11:42:45.720023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:59.090433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.738596+0000","last_change":"2023-10-06T11:42:45.730644+0000","last_active":"2023-10-06T11:42:45.738596+0000","last_peered":"2023-10-06T11:42:45.738596+0000","last_clean":"2023-10-06T11:42:45.738596+0000","last_became_active":"2023-10-06T11:42:45.730388+0000","last_became_peered":"2023-10-06T11:42:45.730388+0000","last_unstale":"2023-10-06T11:42:45.738596+0000","last_undegraded":"2023-10-06T11:42:45.738596+0000","last_fullsized":"2023-10-06T11:42:45.738596+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:45:51.840590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.746984+0000","last_change":"2023-10-06T11:42:44.727309+0000","last_active":"2023-10-06T11:42:45.746984+0000","last_peered":"2023-10-06T11:42:45.746984+0000","last_clean":"2023-10-06T11:42:45.746984+0000","last_became_active":"2023-10-06T11:42:44.727027+0000","last_became_peered":"2023-10-06T11:42:44.727027+0000","last_unstale":"2023-10-06T11:42:45.746984+0000","last_undegraded":"2023-10-06T11:42:45.746984+0000","last_fullsized":"2023-10-06T11:42:45.746984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:34.605521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741456+0000","last_change":"2023-10-06T11:42:45.732002+0000","last_active":"2023-10-06T11:42:45.741456+0000","last_peered":"2023-10-06T11:42:45.741456+0000","last_clean":"2023-10-06T11:42:45.741456+0000","last_became_active":"2023-10-06T11:42:45.731739+0000","last_became_peered":"2023-10-06T11:42:45.731739+0000","last_unstale":"2023-10-06T11:42:45.741456+0000","last_undegraded":"2023-10-06T11:42:45.741456+0000","last_fullsized":"2023-10-06T11:42:45.741456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:57.219243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.752311+0000","last_change":"2023-10-06T11:42:44.729316+0000","last_active":"2023-10-06T11:42:45.752311+0000","last_peered":"2023-10-06T11:42:45.752311+0000","last_clean":"2023-10-06T11:42:45.752311+0000","last_became_active":"2023-10-06T11:42:44.728828+0000","last_became_peered":"2023-10-06T11:42:44.728828+0000","last_unstale":"2023-10-06T11:42:45.752311+0000","last_undegraded":"2023-10-06T11:42:45.752311+0000","last_fullsized":"2023-10-06T11:42:45.752311+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:14:47.724483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.739193+0000","last_change":"2023-10-06T11:42:45.735875+0000","last_active":"2023-10-06T11:42:45.739193+0000","last_peered":"2023-10-06T11:42:45.739193+0000","last_clean":"2023-10-06T11:42:45.739193+0000","last_became_active":"2023-10-06T11:42:45.728544+0000","last_became_peered":"2023-10-06T11:42:45.728544+0000","last_unstale":"2023-10-06T11:42:45.739193+0000","last_undegraded":"2023-10-06T11:42:45.739193+0000","last_fullsized":"2023-10-06T11:42:45.739193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:52.510854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.746962+0000","last_change":"2023-10-06T11:42:44.730399+0000","last_active":"2023-10-06T11:42:45.746962+0000","last_peered":"2023-10-06T11:42:45.746962+0000","last_clean":"2023-10-06T11:42:45.746962+0000","last_became_active":"2023-10-06T11:42:44.730143+0000","last_became_peered":"2023-10-06T11:42:44.730143+0000","last_unstale":"2023-10-06T11:42:45.746962+0000","last_undegraded":"2023-10-06T11:42:45.746962+0000","last_fullsized":"2023-10-06T11:42:45.746962+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:24.544981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749449+0000","last_change":"2023-10-06T11:42:45.740877+0000","last_active":"2023-10-06T11:42:45.749449+0000","last_peered":"2023-10-06T11:42:45.749449+0000","last_clean":"2023-10-06T11:42:45.749449+0000","last_became_active":"2023-10-06T11:42:45.740573+0000","last_became_peered":"2023-10-06T11:42:45.740573+0000","last_unstale":"2023-10-06T11:42:45.749449+0000","last_undegraded":"2023-10-06T11:42:45.749449+0000","last_fullsized":"2023-10-06T11:42:45.749449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.258996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.729237+0000","last_change":"2023-10-06T11:42:44.728344+0000","last_active":"2023-10-06T11:42:45.729237+0000","last_peered":"2023-10-06T11:42:45.729237+0000","last_clean":"2023-10-06T11:42:45.729237+0000","last_became_active":"2023-10-06T11:42:44.728007+0000","last_became_peered":"2023-10-06T11:42:44.728007+0000","last_unstale":"2023-10-06T11:42:45.729237+0000","last_undegraded":"2023-10-06T11:42:45.729237+0000","last_fullsized":"2023-10-06T11:42:45.729237+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.008744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742899+0000","last_change":"2023-10-06T11:42:45.736987+0000","last_active":"2023-10-06T11:42:45.742899+0000","last_peered":"2023-10-06T11:42:45.742899+0000","last_clean":"2023-10-06T11:42:45.742899+0000","last_became_active":"2023-10-06T11:42:45.736707+0000","last_became_peered":"2023-10-06T11:42:45.736707+0000","last_unstale":"2023-10-06T11:42:45.742899+0000","last_undegraded":"2023-10-06T11:42:45.742899+0000","last_fullsized":"2023-10-06T11:42:45.742899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:24.038405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.723988+0000","last_change":"2023-10-06T11:42:44.732425+0000","last_active":"2023-10-06T11:42:45.723988+0000","last_peered":"2023-10-06T11:42:45.723988+0000","last_clean":"2023-10-06T11:42:45.723988+0000","last_became_active":"2023-10-06T11:42:44.732220+0000","last_became_peered":"2023-10-06T11:42:44.732220+0000","last_unstale":"2023-10-06T11:42:45.723988+0000","last_undegraded":"2023-10-06T11:42:45.723988+0000","last_fullsized":"2023-10-06T11:42:45.723988+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:09:54.020473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749383+0000","last_change":"2023-10-06T11:42:45.744706+0000","last_active":"2023-10-06T11:42:45.749383+0000","last_peered":"2023-10-06T11:42:45.749383+0000","last_clean":"2023-10-06T11:42:45.749383+0000","last_became_active":"2023-10-06T11:42:45.743712+0000","last_became_peered":"2023-10-06T11:42:45.743712+0000","last_unstale":"2023-10-06T11:42:45.749383+0000","last_undegraded":"2023-10-06T11:42:45.749383+0000","last_fullsized":"2023-10-06T11:42:45.749383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:41.173527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742502+0000","last_change":"2023-10-06T11:42:44.735752+0000","last_active":"2023-10-06T11:42:45.742502+0000","last_peered":"2023-10-06T11:42:45.742502+0000","last_clean":"2023-10-06T11:42:45.742502+0000","last_became_active":"2023-10-06T11:42:44.735528+0000","last_became_peered":"2023-10-06T11:42:44.735528+0000","last_unstale":"2023-10-06T11:42:45.742502+0000","last_undegraded":"2023-10-06T11:42:45.742502+0000","last_fullsized":"2023-10-06T11:42:45.742502+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:39:22.008744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749238+0000","last_change":"2023-10-06T11:42:45.740827+0000","last_active":"2023-10-06T11:42:45.749238+0000","last_peered":"2023-10-06T11:42:45.749238+0000","last_clean":"2023-10-06T11:42:45.749238+0000","last_became_active":"2023-10-06T11:42:45.740543+0000","last_became_peered":"2023-10-06T11:42:45.740543+0000","last_unstale":"2023-10-06T11:42:45.749238+0000","last_undegraded":"2023-10-06T11:42:45.749238+0000","last_fullsized":"2023-10-06T11:42:45.749238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:33:59.074570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742019+0000","last_change":"2023-10-06T11:42:44.731415+0000","last_active":"2023-10-06T11:42:45.742019+0000","last_peered":"2023-10-06T11:42:45.742019+0000","last_clean":"2023-10-06T11:42:45.742019+0000","last_became_active":"2023-10-06T11:42:44.731100+0000","last_became_peered":"2023-10-06T11:42:44.731100+0000","last_unstale":"2023-10-06T11:42:45.742019+0000","last_undegraded":"2023-10-06T11:42:45.742019+0000","last_fullsized":"2023-10-06T11:42:45.742019+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:35.927136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.732443+0000","last_change":"2023-10-06T11:42:45.727422+0000","last_active":"2023-10-06T11:42:45.732443+0000","last_peered":"2023-10-06T11:42:45.732443+0000","last_clean":"2023-10-06T11:42:45.732443+0000","last_became_active":"2023-10-06T11:42:45.727106+0000","last_became_peered":"2023-10-06T11:42:45.727106+0000","last_unstale":"2023-10-06T11:42:45.732443+0000","last_undegraded":"2023-10-06T11:42:45.732443+0000","last_fullsized":"2023-10-06T11:42:45.732443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:32:33.819762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.740671+0000","last_change":"2023-10-06T11:42:44.728256+0000","last_active":"2023-10-06T11:42:45.740671+0000","last_peered":"2023-10-06T11:42:45.740671+0000","last_clean":"2023-10-06T11:42:45.740671+0000","last_became_active":"2023-10-06T11:42:44.727945+0000","last_became_peered":"2023-10-06T11:42:44.727945+0000","last_unstale":"2023-10-06T11:42:45.740671+0000","last_undegraded":"2023-10-06T11:42:45.740671+0000","last_fullsized":"2023-10-06T11:42:45.740671+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:22:01.511084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741723+0000","last_change":"2023-10-06T11:42:45.733516+0000","last_active":"2023-10-06T11:42:45.741723+0000","last_peered":"2023-10-06T11:42:45.741723+0000","last_clean":"2023-10-06T11:42:45.741723+0000","last_became_active":"2023-10-06T11:42:45.733131+0000","last_became_peered":"2023-10-06T11:42:45.733131+0000","last_unstale":"2023-10-06T11:42:45.741723+0000","last_undegraded":"2023-10-06T11:42:45.741723+0000","last_fullsized":"2023-10-06T11:42:45.741723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:00.587823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749440+0000","last_change":"2023-10-06T11:42:44.728421+0000","last_active":"2023-10-06T11:42:45.749440+0000","last_peered":"2023-10-06T11:42:45.749440+0000","last_clean":"2023-10-06T11:42:45.749440+0000","last_became_active":"2023-10-06T11:42:44.728152+0000","last_became_peered":"2023-10-06T11:42:44.728152+0000","last_unstale":"2023-10-06T11:42:45.749440+0000","last_undegraded":"2023-10-06T11:42:45.749440+0000","last_fullsized":"2023-10-06T11:42:45.749440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T23:37:15.231524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749365+0000","last_change":"2023-10-06T11:42:45.748689+0000","last_active":"2023-10-06T11:42:45.749365+0000","last_peered":"2023-10-06T11:42:45.749365+0000","last_clean":"2023-10-06T11:42:45.749365+0000","last_became_active":"2023-10-06T11:42:45.748407+0000","last_became_peered":"2023-10-06T11:42:45.748407+0000","last_unstale":"2023-10-06T11:42:45.749365+0000","last_undegraded":"2023-10-06T11:42:45.749365+0000","last_fullsized":"2023-10-06T11:42:45.749365+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:25.557367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.730591+0000","last_change":"2023-10-06T11:42:44.732310+0000","last_active":"2023-10-06T11:42:45.730591+0000","last_peered":"2023-10-06T11:42:45.730591+0000","last_clean":"2023-10-06T11:42:45.730591+0000","last_became_active":"2023-10-06T11:42:44.731988+0000","last_became_peered":"2023-10-06T11:42:44.731988+0000","last_unstale":"2023-10-06T11:42:45.730591+0000","last_undegraded":"2023-10-06T11:42:45.730591+0000","last_fullsized":"2023-10-06T11:42:45.730591+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:35.927136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.742912+0000","last_change":"2023-10-06T11:42:45.738171+0000","last_active":"2023-10-06T11:42:45.742912+0000","last_peered":"2023-10-06T11:42:45.742912+0000","last_clean":"2023-10-06T11:42:45.742912+0000","last_became_active":"2023-10-06T11:42:45.737623+0000","last_became_peered":"2023-10-06T11:42:45.737623+0000","last_unstale":"2023-10-06T11:42:45.742912+0000","last_undegraded":"2023-10-06T11:42:45.742912+0000","last_fullsized":"2023-10-06T11:42:45.742912+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:02.157089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.747450+0000","last_change":"2023-10-06T11:42:44.730545+0000","last_active":"2023-10-06T11:42:45.747450+0000","last_peered":"2023-10-06T11:42:45.747450+0000","last_clean":"2023-10-06T11:42:45.747450+0000","last_became_active":"2023-10-06T11:42:44.730255+0000","last_became_peered":"2023-10-06T11:42:44.730255+0000","last_unstale":"2023-10-06T11:42:45.747450+0000","last_undegraded":"2023-10-06T11:42:45.747450+0000","last_fullsized":"2023-10-06T11:42:45.747450+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T19:06:33.028929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.738813+0000","last_change":"2023-10-06T11:42:45.732030+0000","last_active":"2023-10-06T11:42:45.738813+0000","last_peered":"2023-10-06T11:42:45.738813+0000","last_clean":"2023-10-06T11:42:45.738813+0000","last_became_active":"2023-10-06T11:42:45.731178+0000","last_became_peered":"2023-10-06T11:42:45.731178+0000","last_unstale":"2023-10-06T11:42:45.738813+0000","last_undegraded":"2023-10-06T11:42:45.738813+0000","last_fullsized":"2023-10-06T11:42:45.738813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:40.606457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.720006+0000","last_change":"2023-10-06T11:42:44.734861+0000","last_active":"2023-10-06T11:42:45.720006+0000","last_peered":"2023-10-06T11:42:45.720006+0000","last_clean":"2023-10-06T11:42:45.720006+0000","last_became_active":"2023-10-06T11:42:44.734596+0000","last_became_peered":"2023-10-06T11:42:44.734596+0000","last_unstale":"2023-10-06T11:42:45.720006+0000","last_undegraded":"2023-10-06T11:42:45.720006+0000","last_fullsized":"2023-10-06T11:42:45.720006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:48:41.588797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.736474+0000","last_change":"2023-10-06T11:42:45.734231+0000","last_active":"2023-10-06T11:42:45.736474+0000","last_peered":"2023-10-06T11:42:45.736474+0000","last_clean":"2023-10-06T11:42:45.736474+0000","last_became_active":"2023-10-06T11:42:45.733972+0000","last_became_peered":"2023-10-06T11:42:45.733972+0000","last_unstale":"2023-10-06T11:42:45.736474+0000","last_undegraded":"2023-10-06T11:42:45.736474+0000","last_fullsized":"2023-10-06T11:42:45.736474+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:32.402359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.722985+0000","last_change":"2023-10-06T11:42:44.736449+0000","last_active":"2023-10-06T11:42:45.722985+0000","last_peered":"2023-10-06T11:42:45.722985+0000","last_clean":"2023-10-06T11:42:45.722985+0000","last_became_active":"2023-10-06T11:42:44.736174+0000","last_became_peered":"2023-10-06T11:42:44.736174+0000","last_unstale":"2023-10-06T11:42:45.722985+0000","last_undegraded":"2023-10-06T11:42:45.722985+0000","last_fullsized":"2023-10-06T11:42:45.722985+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:24.552817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749211+0000","last_change":"2023-10-06T11:42:45.743999+0000","last_active":"2023-10-06T11:42:45.749211+0000","last_peered":"2023-10-06T11:42:45.749211+0000","last_clean":"2023-10-06T11:42:45.749211+0000","last_became_active":"2023-10-06T11:42:45.740000+0000","last_became_peered":"2023-10-06T11:42:45.740000+0000","last_unstale":"2023-10-06T11:42:45.749211+0000","last_undegraded":"2023-10-06T11:42:45.749211+0000","last_fullsized":"2023-10-06T11:42:45.749211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:24.038405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.719599+0000","last_change":"2023-10-06T11:42:44.733510+0000","last_active":"2023-10-06T11:42:45.719599+0000","last_peered":"2023-10-06T11:42:45.719599+0000","last_clean":"2023-10-06T11:42:45.719599+0000","last_became_active":"2023-10-06T11:42:44.733156+0000","last_became_peered":"2023-10-06T11:42:44.733156+0000","last_unstale":"2023-10-06T11:42:45.719599+0000","last_undegraded":"2023-10-06T11:42:45.719599+0000","last_fullsized":"2023-10-06T11:42:45.719599+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:09.499866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.746936+0000","last_change":"2023-10-06T11:42:45.743154+0000","last_active":"2023-10-06T11:42:45.746936+0000","last_peered":"2023-10-06T11:42:45.746936+0000","last_clean":"2023-10-06T11:42:45.746936+0000","last_became_active":"2023-10-06T11:42:45.742914+0000","last_became_peered":"2023-10-06T11:42:45.742914+0000","last_unstale":"2023-10-06T11:42:45.746936+0000","last_undegraded":"2023-10-06T11:42:45.746936+0000","last_fullsized":"2023-10-06T11:42:45.746936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:16.370379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.718718+0000","last_change":"2023-10-06T11:42:44.730550+0000","last_active":"2023-10-06T11:42:45.718718+0000","last_peered":"2023-10-06T11:42:45.718718+0000","last_clean":"2023-10-06T11:42:45.718718+0000","last_became_active":"2023-10-06T11:42:44.730275+0000","last_became_peered":"2023-10-06T11:42:44.730275+0000","last_unstale":"2023-10-06T11:42:45.718718+0000","last_undegraded":"2023-10-06T11:42:45.718718+0000","last_fullsized":"2023-10-06T11:42:45.718718+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:45:39.601908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.718691+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:42:45.718691+0000","last_peered":"2023-10-06T11:42:45.718691+0000","last_clean":"2023-10-06T11:42:45.718691+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:42:45.718691+0000","last_undegraded":"2023-10-06T11:42:45.718691+0000","last_fullsized":"2023-10-06T11:42:45.718691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.737416+0000","last_change":"2023-10-06T11:42:45.735076+0000","last_active":"2023-10-06T11:42:45.737416+0000","last_peered":"2023-10-06T11:42:45.737416+0000","last_clean":"2023-10-06T11:42:45.737416+0000","last_became_active":"2023-10-06T11:42:45.734800+0000","last_became_peered":"2023-10-06T11:42:45.734800+0000","last_unstale":"2023-10-06T11:42:45.737416+0000","last_undegraded":"2023-10-06T11:42:45.737416+0000","last_fullsized":"2023-10-06T11:42:45.737416+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:49.642325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.727133+0000","last_change":"2023-10-06T11:42:44.727561+0000","last_active":"2023-10-06T11:42:45.727133+0000","last_peered":"2023-10-06T11:42:45.727133+0000","last_clean":"2023-10-06T11:42:45.727133+0000","last_became_active":"2023-10-06T11:42:44.727150+0000","last_became_peered":"2023-10-06T11:42:44.727150+0000","last_unstale":"2023-10-06T11:42:45.727133+0000","last_undegraded":"2023-10-06T11:42:45.727133+0000","last_fullsized":"2023-10-06T11:42:45.727133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:08:42.731902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.726766+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:42:45.726766+0000","last_peered":"2023-10-06T11:42:45.726766+0000","last_clean":"2023-10-06T11:42:45.726766+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:42:45.726766+0000","last_undegraded":"2023-10-06T11:42:45.726766+0000","last_fullsized":"2023-10-06T11:42:45.726766+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.733161+0000","last_change":"2023-10-06T11:42:45.728749+0000","last_active":"2023-10-06T11:42:45.733161+0000","last_peered":"2023-10-06T11:42:45.733161+0000","last_clean":"2023-10-06T11:42:45.733161+0000","last_became_active":"2023-10-06T11:42:45.728465+0000","last_became_peered":"2023-10-06T11:42:45.728465+0000","last_unstale":"2023-10-06T11:42:45.733161+0000","last_undegraded":"2023-10-06T11:42:45.733161+0000","last_fullsized":"2023-10-06T11:42:45.733161+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:41:50.851774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.737011+0000","last_change":"2023-10-06T11:42:44.728433+0000","last_active":"2023-10-06T11:42:45.737011+0000","last_peered":"2023-10-06T11:42:45.737011+0000","last_clean":"2023-10-06T11:42:45.737011+0000","last_became_active":"2023-10-06T11:42:44.728154+0000","last_became_peered":"2023-10-06T11:42:44.728154+0000","last_unstale":"2023-10-06T11:42:45.737011+0000","last_undegraded":"2023-10-06T11:42:45.737011+0000","last_fullsized":"2023-10-06T11:42:45.737011+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:24.552817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.729967+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:42:45.729967+0000","last_peered":"2023-10-06T11:42:45.729967+0000","last_clean":"2023-10-06T11:42:45.729967+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:42:45.729967+0000","last_undegraded":"2023-10-06T11:42:45.729967+0000","last_fullsized":"2023-10-06T11:42:45.729967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.723033+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:42:45.723033+0000","last_peered":"2023-10-06T11:42:45.723033+0000","last_clean":"2023-10-06T11:42:45.723033+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:42:45.723033+0000","last_undegraded":"2023-10-06T11:42:45.723033+0000","last_fullsized":"2023-10-06T11:42:45.723033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.741713+0000","last_change":"2023-10-06T11:42:45.736126+0000","last_active":"2023-10-06T11:42:45.741713+0000","last_peered":"2023-10-06T11:42:45.741713+0000","last_clean":"2023-10-06T11:42:45.741713+0000","last_became_active":"2023-10-06T11:42:45.735858+0000","last_became_peered":"2023-10-06T11:42:45.735858+0000","last_unstale":"2023-10-06T11:42:45.741713+0000","last_undegraded":"2023-10-06T11:42:45.741713+0000","last_fullsized":"2023-10-06T11:42:45.741713+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:13.258996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-10-06T11:42:47.567239+0000","last_change":"2023-10-06T11:42:44.732428+0000","last_active":"2023-10-06T11:42:47.567239+0000","last_peered":"2023-10-06T11:42:47.567239+0000","last_clean":"2023-10-06T11:42:47.567239+0000","last_became_active":"2023-10-06T11:42:44.732122+0000","last_became_peered":"2023-10-06T11:42:44.732122+0000","last_unstale":"2023-10-06T11:42:47.567239+0000","last_undegraded":"2023-10-06T11:42:47.567239+0000","last_fullsized":"2023-10-06T11:42:47.567239+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:12:31.541754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.723495+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:42:45.723495+0000","last_peered":"2023-10-06T11:42:45.723495+0000","last_clean":"2023-10-06T11:42:45.723495+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:42:45.723495+0000","last_undegraded":"2023-10-06T11:42:45.723495+0000","last_fullsized":"2023-10-06T11:42:45.723495+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749399+0000","last_change":"2023-10-06T11:42:45.746531+0000","last_active":"2023-10-06T11:42:45.749399+0000","last_peered":"2023-10-06T11:42:45.749399+0000","last_clean":"2023-10-06T11:42:45.749399+0000","last_became_active":"2023-10-06T11:42:45.746312+0000","last_became_peered":"2023-10-06T11:42:45.746312+0000","last_unstale":"2023-10-06T11:42:45.749399+0000","last_undegraded":"2023-10-06T11:42:45.749399+0000","last_fullsized":"2023-10-06T11:42:45.749399+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:41.173527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.722857+0000","last_change":"2023-10-06T11:42:44.731278+0000","last_active":"2023-10-06T11:42:45.722857+0000","last_peered":"2023-10-06T11:42:45.722857+0000","last_clean":"2023-10-06T11:42:45.722857+0000","last_became_active":"2023-10-06T11:42:44.730741+0000","last_became_peered":"2023-10-06T11:42:44.730741+0000","last_unstale":"2023-10-06T11:42:45.722857+0000","last_undegraded":"2023-10-06T11:42:45.722857+0000","last_fullsized":"2023-10-06T11:42:45.722857+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:34.605521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.722908+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:42:45.722908+0000","last_peered":"2023-10-06T11:42:45.722908+0000","last_clean":"2023-10-06T11:42:45.722908+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:42:45.722908+0000","last_undegraded":"2023-10-06T11:42:45.722908+0000","last_fullsized":"2023-10-06T11:42:45.722908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.749129+0000","last_change":"2023-10-06T11:42:45.743418+0000","last_active":"2023-10-06T11:42:45.749129+0000","last_peered":"2023-10-06T11:42:45.749129+0000","last_clean":"2023-10-06T11:42:45.749129+0000","last_became_active":"2023-10-06T11:42:45.743145+0000","last_became_peered":"2023-10-06T11:42:45.743145+0000","last_unstale":"2023-10-06T11:42:45.749129+0000","last_undegraded":"2023-10-06T11:42:45.749129+0000","last_fullsized":"2023-10-06T11:42:45.749129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T18:57:32.402359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.717554+0000","last_change":"2023-10-06T11:42:44.726717+0000","last_active":"2023-10-06T11:42:45.717554+0000","last_peered":"2023-10-06T11:42:45.717554+0000","last_clean":"2023-10-06T11:42:45.717554+0000","last_became_active":"2023-10-06T11:42:44.726429+0000","last_became_peered":"2023-10-06T11:42:44.726429+0000","last_unstale":"2023-10-06T11:42:45.717554+0000","last_undegraded":"2023-10-06T11:42:45.717554+0000","last_fullsized":"2023-10-06T11:42:45.717554+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T21:38:24.552817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.720992+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:42:45.720992+0000","last_peered":"2023-10-06T11:42:45.720992+0000","last_clean":"2023-10-06T11:42:45.720992+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:42:45.720992+0000","last_undegraded":"2023-10-06T11:42:45.720992+0000","last_fullsized":"2023-10-06T11:42:45.720992+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.743806+0000","last_change":"2023-10-06T11:42:45.734995+0000","last_active":"2023-10-06T11:42:45.743806+0000","last_peered":"2023-10-06T11:42:45.743806+0000","last_clean":"2023-10-06T11:42:45.743806+0000","last_became_active":"2023-10-06T11:42:45.734761+0000","last_became_peered":"2023-10-06T11:42:45.734761+0000","last_unstale":"2023-10-06T11:42:45.743806+0000","last_undegraded":"2023-10-06T11:42:45.743806+0000","last_fullsized":"2023-10-06T11:42:45.743806+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:31:26.001946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.725594+0000","last_change":"2023-10-06T11:42:44.735018+0000","last_active":"2023-10-06T11:42:45.725594+0000","last_peered":"2023-10-06T11:42:45.725594+0000","last_clean":"2023-10-06T11:42:45.725594+0000","last_became_active":"2023-10-06T11:42:44.734735+0000","last_became_peered":"2023-10-06T11:42:44.734735+0000","last_unstale":"2023-10-06T11:42:45.725594+0000","last_undegraded":"2023-10-06T11:42:45.725594+0000","last_fullsized":"2023-10-06T11:42:45.725594+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:29:36.917258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.722822+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:42:45.722822+0000","last_peered":"2023-10-06T11:42:45.722822+0000","last_clean":"2023-10-06T11:42:45.722822+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:42:45.722822+0000","last_undegraded":"2023-10-06T11:42:45.722822+0000","last_fullsized":"2023-10-06T11:42:45.722822+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.739216+0000","last_change":"2023-10-06T11:42:45.736554+0000","last_active":"2023-10-06T11:42:45.739216+0000","last_peered":"2023-10-06T11:42:45.739216+0000","last_clean":"2023-10-06T11:42:45.739216+0000","last_became_active":"2023-10-06T11:42:45.736193+0000","last_became_peered":"2023-10-06T11:42:45.736193+0000","last_unstale":"2023-10-06T11:42:45.739216+0000","last_undegraded":"2023-10-06T11:42:45.739216+0000","last_fullsized":"2023-10-06T11:42:45.739216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:44.708503+0000","last_clean_scrub_stamp":"2023-10-06T11:42:44.708503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:35:24.038405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.718458+0000","last_change":"2023-10-06T11:42:44.735666+0000","last_active":"2023-10-06T11:42:45.718458+0000","last_peered":"2023-10-06T11:42:45.718458+0000","last_clean":"2023-10-06T11:42:45.718458+0000","last_became_active":"2023-10-06T11:42:44.735431+0000","last_became_peered":"2023-10-06T11:42:44.735431+0000","last_unstale":"2023-10-06T11:42:45.718458+0000","last_undegraded":"2023-10-06T11:42:45.718458+0000","last_fullsized":"2023-10-06T11:42:45.718458+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:42:43.703953+0000","last_clean_scrub_stamp":"2023-10-06T11:42:43.703953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:29:06.678801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T11:42:45.723377+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:42:45.723377+0000","last_peered":"2023-10-06T11:42:45.723377+0000","last_clean":"2023-10-06T11:42:45.723377+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:42:45.723377+0000","last_undegraded":"2023-10-06T11:42:45.723377+0000","last_fullsized":"2023-10-06T11:42:45.723377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":738129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":145731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.439},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41699999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":736613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297640,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074200,"statfs":{"total":96636764160,"available":96331980800,"internally_reserved":0,"allocated":360448,"data_stored":145211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.313},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.49299999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.48399999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.52600000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.53800000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.51500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":372736,"data_stored":146747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":146731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":146267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:42:50.259 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T11:42:50.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T11:42:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.385+0000 7f30f0a80700 1 Processor -- start 2023-10-06T11:42:50.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.385+0000 7f30f0a80700 1 -- start start 2023-10-06T11:42:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.385+0000 7f30f0a80700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30ec0cb0b0 0x7f30ec0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.385+0000 7f30f0a80700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec0ccbe0 con 0x7f30ec15bba0 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.385+0000 7f30f0a80700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ec0cc2d0 con 0x7f30ec0cd070 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30f0a80700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ec0ca160 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30ea59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30ec0cb0b0 0x7f30ec0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e9d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f30ec15bba0 0x7f30ec0c8b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36298/0 (socket says 172.21.15.16:36298) 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e9d9b700 1 -- 172.21.15.16:0/3447133844 learned_addr learned my addr 172.21.15.16:0/3447133844 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30ea59c700 1 -- 172.21.15.16:0/3447133844 >> v1:172.21.15.177:6789/0 conn(0x7f30ec0cd070 legacy=0x7f30ec15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30ea59c700 1 -- 172.21.15.16:0/3447133844 >> v1:172.21.15.16:6789/0 conn(0x7f30ec15bba0 legacy=0x7f30ec0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e959a700 1 -- 172.21.15.16:0/3447133844 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988120220 0 0) 0x7f30ec0ccbe0 con 0x7f30ec15bba0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30ea59c700 1 -- 172.21.15.16:0/3447133844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec0c9310 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30ea59c700 1 --2- 172.21.15.16:0/3447133844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec0cb480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009e60 tx=0x7f30e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d17e989056bf247 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e959a700 1 -- 172.21.15.16:0/3447133844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e400e070 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e959a700 1 -- 172.21.15.16:0/3447133844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e400bc30 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.386+0000 7f30e959a700 1 -- 172.21.15.16:0/3447133844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4012430 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 -- 172.21.15.16:0/3447133844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0cb0b0 msgr2=0x7f30ec0cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3447133844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec0cb480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009e60 tx=0x7f30e4009780 comp rx=0 tx=0).stop 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 -- 172.21.15.16:0/3447133844 shutdown_connections 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3447133844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec0cb480 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 -- 172.21.15.16:0/3447133844 >> 172.21.15.16:0/3447133844 conn(0x7f30ec067c90 msgr2=0x7f30ec068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 -- 172.21.15.16:0/3447133844 shutdown_connections 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.387+0000 7f30f0a80700 1 -- 172.21.15.16:0/3447133844 wait complete. 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.388+0000 7f30f0a80700 1 Processor -- start 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.388+0000 7f30f0a80700 1 -- start start 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ec0cd070 0x7f30ec0c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c82e0 con 0x7f30ec15bba0 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c76d0 con 0x7f30ec0cb0b0 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ec0c79b0 con 0x7f30ec0cd070 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47584/0 (socket says 172.21.15.16:47584) 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 -- 172.21.15.16:0/3292875973 learned_addr learned my addr 172.21.15.16:0/3292875973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ea59c700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30e9d9b700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ec0cd070 0x7f30ec0c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 -- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ec0cd070 msgr2=0x7f30ec0c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ec0cd070 0x7f30ec0c3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 -- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ec0cb0b0 msgr2=0x7f30ec14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 -- 172.21.15.16:0/3292875973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4009a20 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30ead9d700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c560 tx=0x7f30dc007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc011590 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc0116f0 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec0c7c90 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.389+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc011ab0 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.390+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ec0c8770 con 0x7f30ec15bba0 2023-10-06T11:42:50.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.390+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ec0c8960 con 0x7f30ec15bba0 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.391+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30dc01b3b0 con 0x7f30ec15bba0 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.391+0000 7f30e27fc700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 0x7f30cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.391+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f30dc083a50 con 0x7f30ec15bba0 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.391+0000 7f30ea59c700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 0x7f30cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.392+0000 7f30ea59c700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 0x7f30cc0693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009e60 tx=0x7f30e4019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.395+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30dc050b60 con 0x7f30ec15bba0 2023-10-06T11:42:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.671+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f30ec04c380 con 0x7f30ec15bba0 2023-10-06T11:42:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.675+0000 7f30e27fc700 1 -- 172.21.15.16:0/3292875973 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f30dc017070 con 0x7f30ec15bba0 2023-10-06T11:42:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 msgr2=0x7f30cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 0x7f30cc0693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009e60 tx=0x7f30e4019000 comp rx=0 tx=0).stop 2023-10-06T11:42:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 msgr2=0x7f30ec0c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c560 tx=0x7f30dc007ef0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 shutdown_connections 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30ec15bba0 0x7f30ec0c4190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f30cc066f30 0x7f30cc0693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f30ec0cb0b0 0x7f30ec14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 --2- 172.21.15.16:0/3292875973 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f30ec0cd070 0x7f30ec0c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 >> 172.21.15.16:0/3292875973 conn(0x7f30ec067c90 msgr2=0x7f30ec1561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 shutdown_connections 2023-10-06T11:42:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.677+0000 7f30f0a80700 1 -- 172.21.15.16:0/3292875973 wait complete. 2023-10-06T11:42:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-10-06T11:42:50.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:42:50.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.808+0000 7f02bff6b700 1 Processor -- start 2023-10-06T11:42:50.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.808+0000 7f02bff6b700 1 -- start start 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bff6b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b80ceb40 0x7f02b8162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bff6b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80cdc90 con 0x7f02b8157750 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bff6b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b80ad920 con 0x7f02b8157380 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bff6b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b80ce120 con 0x7f02b80ceb40 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02be508700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b80ceb40 0x7f02b8162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bd506700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02b8157750 0x7f02b80ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36300/0 (socket says 172.21.15.16:36300) 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02be508700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02b80ceb40 0x7f02b8162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47594/0 (socket says 172.21.15.16:47594) 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.809+0000 7f02bd506700 1 -- 172.21.15.16:0/2588588281 learned_addr learned my addr 172.21.15.16:0/2588588281 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02be508700 1 -- 172.21.15.16:0/2588588281 >> v1:172.21.15.177:6789/0 conn(0x7f02b8157380 legacy=0x7f02b80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02be508700 1 -- 172.21.15.16:0/2588588281 >> v1:172.21.15.16:6789/0 conn(0x7f02b8157750 legacy=0x7f02b80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02be508700 1 -- 172.21.15.16:0/2588588281 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80cd080 con 0x7f02b80ceb40 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02be508700 1 --2- 172.21.15.16:0/2588588281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b8162110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e60 tx=0x7f02a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e787d498ef117edb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02bcd05700 1 -- 172.21.15.16:0/2588588281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800e070 con 0x7f02b80ceb40 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02bcd05700 1 -- 172.21.15.16:0/2588588281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a800bb30 con 0x7f02b80ceb40 2023-10-06T11:42:50.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02bcd05700 1 -- 172.21.15.16:0/2588588281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a800bed0 con 0x7f02b80ceb40 2023-10-06T11:42:50.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02bff6b700 1 -- 172.21.15.16:0/2588588281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 msgr2=0x7f02b8162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.810+0000 7f02bff6b700 1 --2- 172.21.15.16:0/2588588281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b8162110 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009e60 tx=0x7f02a8009780 comp rx=0 tx=0).stop 2023-10-06T11:42:50.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.811+0000 7f02bff6b700 1 -- 172.21.15.16:0/2588588281 shutdown_connections 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.811+0000 7f02bff6b700 1 --2- 172.21.15.16:0/2588588281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b8162110 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.811+0000 7f02bff6b700 1 -- 172.21.15.16:0/2588588281 >> 172.21.15.16:0/2588588281 conn(0x7f02b8067e60 msgr2=0x7f02b8068260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.811+0000 7f02bff6b700 1 -- 172.21.15.16:0/2588588281 shutdown_connections 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.811+0000 7f02bff6b700 1 -- 172.21.15.16:0/2588588281 wait complete. 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.812+0000 7f02bff6b700 1 Processor -- start 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.812+0000 7f02bff6b700 1 -- start start 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.812+0000 7f02bff6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.812+0000 7f02bff6b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8157380 0x7f02b80c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.812+0000 7f02bff6b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b8157750 0x7f02b81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bff6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b816b770 con 0x7f02b80ceb40 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bff6b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b816aba0 con 0x7f02b8157380 2023-10-06T11:42:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bff6b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b816ae40 con 0x7f02b8157750 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47602/0 (socket says 172.21.15.16:47602) 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 -- 172.21.15.16:0/1254305620 learned_addr learned my addr 172.21.15.16:0/1254305620 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bd506700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8157380 0x7f02b80c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02be508700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b8157750 0x7f02b81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 -- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b8157750 msgr2=0x7f02b81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b8157750 0x7f02b81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 -- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8157380 msgr2=0x7f02b80c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8157380 0x7f02b80c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 -- 172.21.15.16:0/1254305620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009a20 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bdd07700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c7c0 tx=0x7f02b0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011590 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0011710 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b816b120 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.813+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011ad0 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.814+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b815ab90 con 0x7f02b80ceb40 2023-10-06T11:42:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.815+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b804c380 con 0x7f02b80ceb40 2023-10-06T11:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.815+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02b001b3b0 con 0x7f02b80ceb40 2023-10-06T11:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.815+0000 7f02b5ffb700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 0x7f02a4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.816+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f02b0083a10 con 0x7f02b80ceb40 2023-10-06T11:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.816+0000 7f02bd506700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 0x7f02a4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.816+0000 7f02bd506700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 0x7f02a4069410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f02b80b2790 tx=0x7f02ac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:50.820+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02b0050b30 con 0x7f02b80ceb40 2023-10-06T11:42:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.056+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02b80cd360 con 0x7f02b80ceb40 2023-10-06T11:42:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.056+0000 7f02b5ffb700 1 -- 172.21.15.16:0/1254305620 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f02b0017070 con 0x7f02b80ceb40 2023-10-06T11:42:51.058 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:42:51.059 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 msgr2=0x7f02a4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 0x7f02a4069410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f02b80b2790 tx=0x7f02ac009040 comp rx=0 tx=0).stop 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 msgr2=0x7f02b815a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c7c0 tx=0x7f02b0007f70 comp rx=0 tx=0).stop 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 shutdown_connections 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02b80ceb40 0x7f02b815a5d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f02a4066f60 0x7f02a4069410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02b8157750 0x7f02b81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 --2- 172.21.15.16:0/1254305620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02b8157380 0x7f02b80c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 >> 172.21.15.16:0/1254305620 conn(0x7f02b8067e60 msgr2=0x7f02b8153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 shutdown_connections 2023-10-06T11:42:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.058+0000 7f02bff6b700 1 -- 172.21.15.16:0/1254305620 wait complete. 2023-10-06T11:42:51.062 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 17 2023-10-06T11:42:51.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T11:42:45.120937+0000', 'modified': '2023-10-06T11:42:48.742791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4977}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4977': {'gid': 4977, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/1696671501', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 1696671501}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 1696671501}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T11:42:51.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T11:42:51.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T11:42:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.599+0000 7f32d6fd6700 1 Processor -- start 2023-10-06T11:42:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.599+0000 7f32d6fd6700 1 -- start start 2023-10-06T11:42:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32d6fd6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 0x7f32d00cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32d6fd6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00cf060 con 0x7f32d006cbf0 2023-10-06T11:42:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32d6fd6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00cf900 con 0x7f32d00cd8a0 2023-10-06T11:42:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32d6fd6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0154580 con 0x7f32d015ab50 2023-10-06T11:42:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32cffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32d006cbf0 0x7f32d00cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36306/0 (socket says 172.21.15.16:36306) 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32cffff700 1 -- 172.21.15.16:0/1024299123 learned_addr learned my addr 172.21.15.16:0/1024299123 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.600+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1024299123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 0x7f32d00cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887329110 0 0) 0x7f32d00cf060 con 0x7f32d006cbf0 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b8003570 con 0x7f32d006cbf0 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805015108 0 0) 0x7f32d00cf900 con 0x7f32d00cd8a0 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d00cf060 con 0x7f32d00cd8a0 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32d4d72700 1 -- 172.21.15.16:0/1024299123 >> v1:172.21.15.177:6789/0 conn(0x7f32d00cd8a0 legacy=0x7f32d00cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32d4d72700 1 -- 172.21.15.16:0/1024299123 >> v1:172.21.15.16:6789/0 conn(0x7f32d006cbf0 legacy=0x7f32d00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32d4d72700 1 -- 172.21.15.16:0/1024299123 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00cdec0 con 0x7f32d015ab50 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1024299123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 0x7f32d00cfd90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009900 tx=0x7f32c0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=412eb6e1bf178b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0005c10 con 0x7f32d015ab50 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0005d70 con 0x7f32d015ab50 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.601+0000 7f32cf7fe700 1 -- 172.21.15.16:0/1024299123 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0006260 con 0x7f32d015ab50 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 -- 172.21.15.16:0/1024299123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 msgr2=0x7f32d00cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/1024299123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 0x7f32d00cfd90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009900 tx=0x7f32c0009120 comp rx=0 tx=0).stop 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 -- 172.21.15.16:0/1024299123 shutdown_connections 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/1024299123 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d015ab50 0x7f32d00cfd90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 -- 172.21.15.16:0/1024299123 >> 172.21.15.16:0/1024299123 conn(0x7f32d008a270 msgr2=0x7f32d0099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 -- 172.21.15.16:0/1024299123 shutdown_connections 2023-10-06T11:42:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.602+0000 7f32d6fd6700 1 -- 172.21.15.16:0/1024299123 wait complete. 2023-10-06T11:42:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.603+0000 7f32d6fd6700 1 Processor -- start 2023-10-06T11:42:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.603+0000 7f32d6fd6700 1 -- start start 2023-10-06T11:42:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d006cbf0 0x7f32d01550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d00cd8a0 0x7f32d0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0153b60 con 0x7f32d015ab50 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d016a2c0 con 0x7f32d006cbf0 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d6fd6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0156f00 con 0x7f32d00cd8a0 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47612/0 (socket says 172.21.15.16:47612) 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 -- 172.21.15.16:0/2845143183 learned_addr learned my addr 172.21.15.16:0/2845143183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d4d72700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d006cbf0 0x7f32d01550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32cffff700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d00cd8a0 0x7f32d0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 -- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d00cd8a0 msgr2=0x7f32d0153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d00cd8a0 0x7f32d0153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 -- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d006cbf0 msgr2=0x7f32d01550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d006cbf0 0x7f32d01550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 -- 172.21.15.16:0/2845143183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c00093c0 con 0x7f32d015ab50 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.604+0000 7f32d5573700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c970 tx=0x7f32c80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.605+0000 7f32ccff9700 1 -- 172.21.15.16:0/2845143183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8012630 con 0x7f32d015ab50 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.605+0000 7f32ccff9700 1 -- 172.21.15.16:0/2845143183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c80127b0 con 0x7f32d015ab50 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.605+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d0157180 con 0x7f32d015ab50 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.605+0000 7f32ccff9700 1 -- 172.21.15.16:0/2845143183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8012b70 con 0x7f32d015ab50 2023-10-06T11:42:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.605+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0157740 con 0x7f32d015ab50 2023-10-06T11:42:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.606+0000 7f32ccff9700 1 -- 172.21.15.16:0/2845143183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32c801b400 con 0x7f32d015ab50 2023-10-06T11:42:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.606+0000 7f32ccff9700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 0x7f32bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.606+0000 7f32ccff9700 1 -- 172.21.15.16:0/2845143183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32c80849e0 con 0x7f32d015ab50 2023-10-06T11:42:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.607+0000 7f32d4d72700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 0x7f32bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.607+0000 7f32d4d72700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 0x7f32bc0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009390 tx=0x7f32c0016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.608+0000 7f32e02ceb80 1 Processor -- start 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.608+0000 7f32e02ceb80 1 -- start start 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a6184a90 0x5615a6184e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a613dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a60bdc80 con 0x5615a61409a0 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a6094a70 con 0x5615a6184a90 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615a6092270 con 0x5615a613ebd0 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32cffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32cffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47616/0 (socket says 172.21.15.16:47616) 2023-10-06T11:42:51.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32cffff700 1 -- 172.21.15.16:0/4016602235 learned_addr learned my addr 172.21.15.16:0/4016602235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32d4d72700 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a6184a90 0x5615a6184e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32d5573700 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a613dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32cffff700 1 -- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 msgr2=0x5615a613dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.609+0000 7f32cffff700 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a613dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32cffff700 1 -- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a6184a90 msgr2=0x5615a6184e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32cffff700 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a6184a90 0x5615a6184e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32cffff700 1 -- 172.21.15.16:0/4016602235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615a5f28270 con 0x5615a61409a0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32cffff700 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f32c4001790 tx=0x7f32c4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801bde53cad54d96 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32b5ffb700 1 -- 172.21.15.16:0/4016602235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4005b10 con 0x5615a61409a0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32b5ffb700 1 -- 172.21.15.16:0/4016602235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c4005c90 con 0x5615a61409a0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32b5ffb700 1 -- 172.21.15.16:0/4016602235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40072e0 con 0x5615a61409a0 2023-10-06T11:42:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 -- 172.21.15.16:0/4016602235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 msgr2=0x5615a613d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f32c4001790 tx=0x7f32c4009040 comp rx=0 tx=0).stop 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 -- 172.21.15.16:0/4016602235 shutdown_connections 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a613dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a6184a90 0x5615a6184e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/4016602235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a613d580 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.610+0000 7f32e02ceb80 1 -- 172.21.15.16:0/4016602235 >> 172.21.15.16:0/4016602235 conn(0x5615a613f070 msgr2=0x5615a613f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.611+0000 7f32e02ceb80 1 -- 172.21.15.16:0/4016602235 shutdown_connections 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.611+0000 7f32e02ceb80 1 -- 172.21.15.16:0/4016602235 wait complete. 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.611+0000 7f32e02ceb80 1 -- start start 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.611+0000 7f32e02ceb80 1 Processor -- start 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 0x5615a6169b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a613d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a608e060 con 0x5615a61409a0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a6039890 con 0x5615a613ebd0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615a6054ea0 con 0x5615a6184a90 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32e02ceb80 10 client.0 _ll_register_callbacks cb 0x5615a5f2af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47618/0 (socket says 172.21.15.16:47618) 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 -- 172.21.15.16:0/3997804940 learned_addr learned my addr 172.21.15.16:0/3997804940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32d5573700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a613d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32d4d72700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 0x5615a6169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 -- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 msgr2=0x5615a613d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a613d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 -- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 msgr2=0x5615a6169b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 0x5615a6169b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.612+0000 7f32cffff700 1 -- 172.21.15.16:0/3997804940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b8003570 con 0x5615a61409a0 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32d4d72700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 0x5615a6169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32cffff700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32c4014000 tx=0x7f32c4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e732beda4c58784d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4006e90 con 0x5615a61409a0 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32c401d620 con 0x5615a61409a0 2023-10-06T11:42:51.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615a6132690 con 0x5615a61409a0 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c401d9f0 con 0x5615a61409a0 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 10 client.5175 fetch_fsmap learned FSMap version 17 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5615a5ecafe0 con 0x5615a61409a0 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5615a6132690 con 0x5615a61409a0 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f32c4012070 con 0x5615a61409a0 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 10 client.5175 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 10 client.5175 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 20 client.5175 populate_metadata read hostname 'smithi016' 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a6168610 0x5615a61689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 4 client.5175 mds_command: new command op to 4977 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 --> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615a610f5c0 con 0x5615a6168610 2023-10-06T11:42:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.613+0000 7f32d4d72700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a6168610 0x5615a61689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.614+0000 7f32aeffd700 10 client.5175 ms_handle_connect on v2:172.21.15.16:6833/1696671501 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.614+0000 7f32d4d72700 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a6168610 0x5615a61689e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16075baba2099a52 server_cookie=af334a5fe3e560a7 in_seq=0 out_seq=0 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.614+0000 7f32aeffd700 1 -- 172.21.15.16:0/3997804940 <== mds.0 v2:172.21.15.16:6833/1696671501 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5615a610f5c0 con 0x5615a6168610 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.614+0000 7f32aeffd700 10 client.5175 handle_command_reply: tid=0 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.614+0000 7f32e02ceb80 1 client.5175 shutdown 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 msgr2=0x5615a6181ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f32c4014000 tx=0x7f32c4018000 comp rx=0 tx=0).stop 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 shutdown_connections 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a613d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a613ebd0 0x5615a6169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a6168610 0x5615a61689e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/3997804940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a61409a0 0x5615a6181ab0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32aeffd700 0 client.5175 ms_handle_reset on v2:172.21.15.16:6833/1696671501 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 >> 172.21.15.16:0/3997804940 conn(0x5615a613f070 msgr2=0x5615a613a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 shutdown_connections 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 1 -- 172.21.15.16:0/3997804940 wait complete. 2023-10-06T11:42:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.615+0000 7f32e02ceb80 20 client.5175 trim_cache size 0 max 16384 2023-10-06T11:42:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 Processor -- start 2023-10-06T11:42:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 -- start start 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a61689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a5f10e30 con 0x5615a613ebd0 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a619d0c0 con 0x5615a61409a0 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615a611d030 con 0x5615a6184a90 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32d4d72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32d4d72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47624/0 (socket says 172.21.15.16:47624) 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.632+0000 7f32d4d72700 1 -- 172.21.15.16:0/918082874 learned_addr learned my addr 172.21.15.16:0/918082874 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 -- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 msgr2=0x5615a61689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a61689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 -- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 msgr2=0x5615a61e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 -- 172.21.15.16:0/918082874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615a605c780 con 0x5615a613ebd0 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32d4d72700 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006aa0 tx=0x7f32c0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aeece2fdc8f97c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32b67fc700 1 -- 172.21.15.16:0/918082874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0023070 con 0x5615a613ebd0 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32b67fc700 1 -- 172.21.15.16:0/918082874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0032040 con 0x5615a613ebd0 2023-10-06T11:42:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.633+0000 7f32b67fc700 1 -- 172.21.15.16:0/918082874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c00398f0 con 0x5615a613ebd0 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- 172.21.15.16:0/918082874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 msgr2=0x5615a616d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f32c0006aa0 tx=0x7f32c0013000 comp rx=0 tx=0).stop 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- 172.21.15.16:0/918082874 shutdown_connections 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a613ebd0 0x5615a616d700 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a6184a90 0x5615a61689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/918082874 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- 172.21.15.16:0/918082874 >> 172.21.15.16:0/918082874 conn(0x5615a613f070 msgr2=0x5615a6184e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- 172.21.15.16:0/918082874 shutdown_connections 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- 172.21.15.16:0/918082874 wait complete. 2023-10-06T11:42:51.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 -- start start 2023-10-06T11:42:51.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.634+0000 7f32e02ceb80 1 Processor -- start 2023-10-06T11:42:51.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a61d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a618b840 con 0x5615a6184a90 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615a61964c0 con 0x5615a61409a0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5615a6183e90 con 0x5615a613ebd0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32e02ceb80 10 client.0 _ll_register_callbacks cb 0x5615a5f2af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32d5573700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32d5573700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47638/0 (socket says 172.21.15.16:47638) 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32d5573700 1 -- 172.21.15.16:0/1023130499 learned_addr learned my addr 172.21.15.16:0/1023130499 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a61d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.635+0000 7f32cffff700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 -- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 msgr2=0x5615a61d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a61d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 -- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 msgr2=0x5615a61689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 -- 172.21.15.16:0/1023130499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d00cdec0 con 0x5615a6184a90 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a61d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32cffff700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d5573700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32c8055740 tx=0x7f32c8058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8166ad937282b5c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80712c0 con 0x5615a6184a90 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f32c8054ca0 con 0x5615a6184a90 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32c4005c90 con 0x5615a6184a90 2023-10-06T11:42:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32c8071420 con 0x5615a6184a90 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 10 client.5181 fetch_fsmap learned FSMap version 17 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5615a6043470 con 0x5615a6184a90 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8060070 con 0x5615a6184a90 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f32c805c2c0 con 0x5615a6184a90 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 10 client.5181 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 10 client.5181 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 20 client.5181 populate_metadata read hostname 'smithi016' 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a618abd0 0x5615a618afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 4 client.5181 mds_command: new command op to 4977 tid=0[{"prefix": "status"}] 2023-10-06T11:42:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 --> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5615a6030e00 con 0x5615a618abd0 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.636+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a618abd0 0x5615a618afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32aeffd700 10 client.5181 ms_handle_connect on v2:172.21.15.16:6833/1696671501 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32d4d72700 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a618abd0 0x5615a618afa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b10682bb82fde6 server_cookie=6a531e77449f68b4 in_seq=0 out_seq=0 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32aeffd700 1 -- 172.21.15.16:0/1023130499 <== mds.0 v2:172.21.15.16:6833/1696671501 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5615a6030e00 con 0x5615a618abd0 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32aeffd700 10 client.5181 handle_command_reply: tid=0 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 client.5181 shutdown 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 msgr2=0x5615a6168f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f32c8055740 tx=0x7f32c8058040 comp rx=0 tx=0).stop 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 shutdown_connections 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:6833/1696671501,v1:172.21.15.16:6835/1696671501] conn(0x5615a618abd0 0x5615a618afa0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5615a6184a90 0x5615a6168f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5615a613ebd0 0x5615a61d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 --2- 172.21.15.16:0/1023130499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5615a61409a0 0x5615a61689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32aeffd700 0 client.5181 ms_handle_reset on v2:172.21.15.16:6833/1696671501 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 >> 172.21.15.16:0/1023130499 conn(0x5615a613f070 msgr2=0x5615a6184e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 shutdown_connections 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 1 -- 172.21.15.16:0/1023130499 wait complete. 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.637+0000 7f32e02ceb80 20 client.5181 trim_cache size 0 max 16384 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:51.647 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "b7fe6d1b-a342-44e2-9c53-ba3d26f05e41", 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4977, 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 5.9239916109999999, 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 17, 2023-10-06T11:42:51.648 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 32, 2023-10-06T11:42:51.649 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 32, 2023-10-06T11:42:51.649 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 13.800342271 2023-10-06T11:42:51.649 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 msgr2=0x7f32bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 0x7f32bc0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009390 tx=0x7f32c0016000 comp rx=0 tx=0).stop 2023-10-06T11:42:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 msgr2=0x7f32d01569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c970 tx=0x7f32c80027a0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 shutdown_connections 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32d015ab50 0x7f32d01569c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f32bc066f30 0x7f32bc0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f32d006cbf0 0x7f32d01550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 --2- 172.21.15.16:0/2845143183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f32d00cd8a0 0x7f32d0153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 >> 172.21.15.16:0/2845143183 conn(0x7f32d008a270 msgr2=0x7f32d006e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 shutdown_connections 2023-10-06T11:42:51.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.638+0000 7f32d6fd6700 1 -- 172.21.15.16:0/2845143183 wait complete. 2023-10-06T11:42:51.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.770+0000 7fbc0806a700 1 Processor -- start 2023-10-06T11:42:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc0806a700 1 -- start start 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc0806a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc000cd850 0x7fbc000cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc0806a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000cbf20 con 0x7fbc001595b0 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc0806a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc000cc200 con 0x7fbc00159980 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc0806a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc000cc4e0 con 0x7fbc000cd850 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc05605700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc000cd850 0x7fbc000cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.771+0000 7fbc05e06700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbc001595b0 0x7fbc000cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36312/0 (socket says 172.21.15.16:36312) 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc05e06700 1 -- 172.21.15.16:0/1196999720 learned_addr learned my addr 172.21.15.16:0/1196999720 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc04e04700 1 -- 172.21.15.16:0/1196999720 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047501110 0 0) 0x7fbc000cbf20 con 0x7fbc001595b0 2023-10-06T11:42:51.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc04e04700 1 -- 172.21.15.16:0/1196999720 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe8003570 con 0x7fbc001595b0 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc05605700 1 -- 172.21.15.16:0/1196999720 >> v1:172.21.15.177:6789/0 conn(0x7fbc00159980 legacy=0x7fbc000cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc05605700 1 -- 172.21.15.16:0/1196999720 >> v1:172.21.15.16:6789/0 conn(0x7fbc001595b0 legacy=0x7fbc000cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc05605700 1 -- 172.21.15.16:0/1196999720 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc000cc7c0 con 0x7fbc000cd850 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc05605700 1 --2- 172.21.15.16:0/1196999720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc000cec70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40098e0 tx=0x7fbbf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d9da9d6a062d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc04e04700 1 -- 172.21.15.16:0/1196999720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011630 con 0x7fbc000cd850 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc04e04700 1 -- 172.21.15.16:0/1196999720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4011790 con 0x7fbc000cd850 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.772+0000 7fbc04e04700 1 -- 172.21.15.16:0/1196999720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf4011b30 con 0x7fbc000cd850 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 -- 172.21.15.16:0/1196999720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 msgr2=0x7fbc000cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 --2- 172.21.15.16:0/1196999720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc000cec70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40098e0 tx=0x7fbbf40091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 -- 172.21.15.16:0/1196999720 shutdown_connections 2023-10-06T11:42:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 --2- 172.21.15.16:0/1196999720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc000cec70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 -- 172.21.15.16:0/1196999720 >> 172.21.15.16:0/1196999720 conn(0x7fbc00099b40 msgr2=0x7fbc00087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 -- 172.21.15.16:0/1196999720 shutdown_connections 2023-10-06T11:42:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.773+0000 7fbc0806a700 1 -- 172.21.15.16:0/1196999720 wait complete. 2023-10-06T11:42:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.774+0000 7fbc0806a700 1 Processor -- start 2023-10-06T11:42:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.774+0000 7fbc0806a700 1 -- start start 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc001595b0 0x7fbc001699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc00159980 0x7fbc000b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0015cea0 con 0x7fbc000cd850 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000b3530 con 0x7fbc00159980 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc000b3810 con 0x7fbc001595b0 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47646/0 (socket says 172.21.15.16:47646) 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 -- 172.21.15.16:0/4170425294 learned_addr learned my addr 172.21.15.16:0/4170425294 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05605700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc001595b0 0x7fbc001699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc06607700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc00159980 0x7fbc000b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 -- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc001595b0 msgr2=0x7fbc001699b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc001595b0 0x7fbc001699b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 -- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc00159980 msgr2=0x7fbc000b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc00159980 0x7fbc000b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 -- 172.21.15.16:0/4170425294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4009420 con 0x7fbc000cd850 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc05e06700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800cd60 tx=0x7fbbf8001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800ab30 con 0x7fbc000cd850 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf800acb0 con 0x7fbc000cd850 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc00152ee0 con 0x7fbc000cd850 2023-10-06T11:42:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.775+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8007340 con 0x7fbc000cd850 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.776+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0016a190 con 0x7fbc000cd850 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.777+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0004c380 con 0x7fbc000cd850 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.777+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbbf8007730 con 0x7fbc000cd850 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.777+0000 7fbbfdffb700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 0x7fbbec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.777+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbbf8084ba0 con 0x7fbc000cd850 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.778+0000 7fbc05605700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 0x7fbbec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:51.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.778+0000 7fbc05605700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 0x7fbbec0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc000cdc50 tx=0x7fbbf4016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:51.782+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbf8051d30 con 0x7fbc000cd850 2023-10-06T11:42:52.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.020+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc000ccaa0 con 0x7fbc000cd850 2023-10-06T11:42:52.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.021+0000 7fbbfdffb700 1 -- 172.21.15.16:0/4170425294 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbbf8055350 con 0x7fbc000cd850 2023-10-06T11:42:52.023 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 msgr2=0x7fbbec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 0x7fbbec0693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc000cdc50 tx=0x7fbbf4016000 comp rx=0 tx=0).stop 2023-10-06T11:42:52.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 msgr2=0x7fbc001529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800cd60 tx=0x7fbbf8001b10 comp rx=0 tx=0).stop 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 shutdown_connections 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc000cd850 0x7fbc001529a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fbbec066f30 0x7fbbec0693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbc00159980 0x7fbc000b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:52.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 --2- 172.21.15.16:0/4170425294 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbc001595b0 0x7fbc001699b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 >> 172.21.15.16:0/4170425294 conn(0x7fbc00099b40 msgr2=0x7fbc0006dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 shutdown_connections 2023-10-06T11:42:52.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:52.023+0000 7fbc0806a700 1 -- 172.21.15.16:0/4170425294 wait complete. 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:52.037 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:52.038 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:52.038 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:52.038 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:42:52.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.038 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:52.055 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-10-06T11:42:52.056 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42422sec preferred_lft 42422sec 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 06:4e:29:32:47:a3 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.057 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::582b:4fff:fe01:ca4e/64 scope link 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:dd:72:ae:41:b0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::24dd:72ff:feae:41b0/64 scope link 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 06:4e:29:32:47:a3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::44e:29ff:fe32:47a3/64 scope link 2023-10-06T11:42:52.058 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:52.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.059 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:52.119 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-10-06T11:42:52.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.120 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-10-06T11:42:52.176 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-10-06T11:42:52.177 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:52.177 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T11:42:52.177 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T11:42:52.177 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:52.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T11:42:52.178 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-10-06T11:42:52.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.279 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-10-06T11:42:52.294 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:42:52.294 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:42:52.295 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:42:52.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:52.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-10-06T11:42:52.362 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T11:42:52.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:52.362 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:52.417 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T11:42:52.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:52.417 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:52.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:52.537 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:52.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:52.602 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:52.603 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:52.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:52.603 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:52.658 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:42:52.659 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T11:42:52.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-10-06T11:42:52.727 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:52.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:52.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-10-06T11:42:52.785+0000 7f1e408f8540 -1 init, newargv = 0x55cf3cf1d8b0 newargc=15 2023-10-06T11:42:52.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[103256]: starting ceph client 2023-10-06T11:42:52.792 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[103256]: starting fuse 2023-10-06T11:42:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:52.794 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:52.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:52.795 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-10-06T11:42:52.858 INFO:teuthology.orchestra.run.smithi016.stdout:47 2023-10-06T11:42:52.858 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T11:42:52.859 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:52.860 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-10-06T11:42:52.861 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-10-06T11:42:52.983 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.103256.asok 2023-10-06T11:42:53.164 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103256.asok 2023-10-06T11:42:53.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:53.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103256.asok status 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-10-06T11:42:53.320 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "103256", 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5196, 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5196 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:53.321 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 471440540 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 471440540 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5196 192.168.0.1:0/471440540", 2023-10-06T11:42:53.322 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/471440540", 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 17, 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 32, 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-10-06T11:42:53.323 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:42:53.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:53.330 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-10-06T11:42:53.330 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:53.346 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-10-06T11:42:53.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:53.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:53.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T11:42:53.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T11:42:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.570+0000 7fe1caebf700 1 Processor -- start 2023-10-06T11:42:53.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.570+0000 7fe1caebf700 1 -- start start 2023-10-06T11:42:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.570+0000 7fe1caebf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.570+0000 7fe1caebf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 0x7fe1c40cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1caebf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c40cbf20 con 0x7fe1c40cec70 2023-10-06T11:42:53.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1caebf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c40cc200 con 0x7fe1c4159980 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1caebf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40cc4e0 con 0x7fe1c41595b0 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c3fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 0x7fe1c40cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:33096/0 (socket says 172.21.15.16:33096) 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c945c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fe1c40cec70 0x7fe1c40cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:40516/0 (socket says 172.21.15.16:40516) 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 -- 172.21.15.16:0/264288618 learned_addr learned my addr 172.21.15.16:0/264288618 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c3fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 0x7fe1c40cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:55042/0 (socket says 172.21.15.16:55042) 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 -- 172.21.15.16:0/264288618 >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 msgr2=0x7fe1c40cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 --2- 172.21.15.16:0/264288618 >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 0x7fe1c40cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 -- 172.21.15.16:0/264288618 >> v1:172.21.15.177:6789/0 conn(0x7fe1c40cec70 legacy=0x7fe1c40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 -- 172.21.15.16:0/264288618 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40cc7c0 con 0x7fe1c41595b0 2023-10-06T11:42:53.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.571+0000 7fe1c8c5b700 1 --2- 172.21.15.16:0/264288618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b327fbbb41fc75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1c37fe700 1 -- 172.21.15.16:0/264288618 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4009ea0 con 0x7fe1c41595b0 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1c37fe700 1 -- 172.21.15.16:0/264288618 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b400b860 con 0x7fe1c41595b0 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1c37fe700 1 -- 172.21.15.16:0/264288618 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400bc00 con 0x7fe1c41595b0 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 -- 172.21.15.16:0/264288618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 msgr2=0x7fe1c40cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 --2- 172.21.15.16:0/264288618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4009250 comp rx=0 tx=0).stop 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 -- 172.21.15.16:0/264288618 shutdown_connections 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 --2- 172.21.15.16:0/264288618 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c41595b0 0x7fe1c40cd140 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 --2- 172.21.15.16:0/264288618 >> v2:172.21.15.177:3300/0 conn(0x7fe1c4159980 0x7fe1c40cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 -- 172.21.15.16:0/264288618 >> 172.21.15.16:0/264288618 conn(0x7fe1c4099b40 msgr2=0x7fe1c4087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.572+0000 7fe1caebf700 1 -- 172.21.15.16:0/264288618 shutdown_connections 2023-10-06T11:42:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.573+0000 7fe1caebf700 1 -- 172.21.15.16:0/264288618 wait complete. 2023-10-06T11:42:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.574+0000 7fe1caebf700 1 Processor -- start 2023-10-06T11:42:53.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.574+0000 7fe1caebf700 1 -- start start 2023-10-06T11:42:53.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c40cec70 0x7fe1c409b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1c41595b0 0x7fe1c41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40c7820 con 0x7fe1c4159980 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c409bea0 con 0x7fe1c41595b0 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4153700 con 0x7fe1c40cec70 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47652/0 (socket says 172.21.15.16:47652) 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 -- 172.21.15.16:0/3389466753 learned_addr learned my addr 172.21.15.16:0/3389466753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1c41595b0 0x7fe1c41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c8c5b700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c40cec70 0x7fe1c409b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 -- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c40cec70 msgr2=0x7fe1c409b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c40cec70 0x7fe1c409b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 -- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1c41595b0 msgr2=0x7fe1c41408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1c41595b0 0x7fe1c41408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 -- 172.21.15.16:0/3389466753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b40094f0 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c945c700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00cd00 tx=0x7fe1bc001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00ce60 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc005df0 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c409cce0 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.575+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00ab00 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.576+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c409c180 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.576+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c409ce70 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.577+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1bc0070f0 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.577+0000 7fe1c0ff9700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.577+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1bc084740 con 0x7fe1c4159980 2023-10-06T11:42:53.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.577+0000 7fe1c8c5b700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.578+0000 7fe1c8c5b700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 0x7fe1a40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:53.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.582+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1bc051830 con 0x7fe1c4159980 2023-10-06T11:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.822+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1c404c380 con 0x7fe1c4159980 2023-10-06T11:42:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.822+0000 7fe1c0ff9700 1 -- 172.21.15.16:0/3389466753 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe1bc054e50 con 0x7fe1c4159980 2023-10-06T11:42:53.825 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 msgr2=0x7fe1a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 0x7fe1a40693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40099b0 tx=0x7fe1b4017000 comp rx=0 tx=0).stop 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 msgr2=0x7fe1c4155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00cd00 tx=0x7fe1bc001b10 comp rx=0 tx=0).stop 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 shutdown_connections 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4159980 0x7fe1c4155980 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe1c40cec70 0x7fe1c409b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 --2- 172.21.15.16:0/3389466753 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1c41595b0 0x7fe1c41408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 >> 172.21.15.16:0/3389466753 conn(0x7fe1c4099b40 msgr2=0x7fe1c4155190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.824+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 shutdown_connections 2023-10-06T11:42:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:53.825+0000 7fe1caebf700 1 -- 172.21.15.16:0/3389466753 wait complete. 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T11:42:53.839 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:42:53.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:53.840 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T11:42:53.859 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:53.860 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42425sec preferred_lft 42425sec 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether de:02:13:dd:df:15 brd ff:ff:ff:ff:ff:ff 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::3805:bdff:fe6b:f2aa/64 scope link 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether de:02:13:dd:df:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::dc02:13ff:fedd:df15/64 scope link 2023-10-06T11:42:53.861 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.862 INFO:teuthology.orchestra.run.smithi177.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T11:42:53.862 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether de:1a:cb:19:1f:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:42:53.862 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::dc1a:cbff:fe19:1fbb/64 scope link 2023-10-06T11:42:53.862 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T11:42:53.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:53.862 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T11:42:53.928 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T11:42:53.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:53.929 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-10-06T11:42:53.987 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-10-06T11:42:53.987 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T11:42:53.987 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T11:42:53.988 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T11:42:53.988 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T11:42:53.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:53.988 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T11:42:53.988 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T11:42:53.988 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T11:42:53.988 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T11:42:53.989 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T11:42:53.989 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T11:42:54.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:54.093 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T11:42:54.107 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T11:42:54.107 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:42:54.108 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:42:54.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:42:54.108 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2023-10-06T11:42:54.173 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T11:42:54.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:54.173 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:54.228 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T11:42:54.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T11:42:54.228 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:54.282 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:54.346 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:54.346 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:54.412 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:54.413 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:54.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:54.414 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:54.468 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T11:42:54.468 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T11:42:54.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T11:42:54.533 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T11:42:54.533 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T11:42:54.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:2023-10-06T11:42:54.589+0000 7f8e6158f540 -1 init, newargv = 0x5614b72b1cc0 newargc=15 2023-10-06T11:42:54.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[98215]: starting ceph client 2023-10-06T11:42:54.596 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[98215]: starting fuse 2023-10-06T11:42:54.600 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T11:42:54.601 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T11:42:54.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:54.603 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T11:42:54.658 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T11:42:54.658 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> import glob 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> import re 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> import os 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> import subprocess 2023-10-06T11:42:54.658 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> def _find_admin_socket(client_name): 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> files = glob.glob(asok_path) 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> # Given a non-glob path, it better be there 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> if "*" not in asok_path: 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> assert(len(files) == 1) 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> return files[0] 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> for f in files: 2023-10-06T11:42:54.659 DEBUG:teuthology.orchestra.run.smithi177:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> contents = proc_f.read() 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> if mountpoint in contents: 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> return f 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> print(_find_admin_socket("client.1")) 2023-10-06T11:42:54.660 DEBUG:teuthology.orchestra.run.smithi177:> ' 2023-10-06T11:42:54.778 INFO:teuthology.orchestra.run.smithi177.stdout:/var/run/ceph/ceph-client.1.98215.asok 2023-10-06T11:42:54.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.98215.asok 2023-10-06T11:42:54.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:54.960 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.98215.asok status 2023-10-06T11:42:55.111 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "metadata": { 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_sha1": "68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 17.2.6-1274-g68e53a4a (68e53a4a7d4ae06d4fc1a0a7cfef840d1d942564) quincy (stable)", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "entity_id": "1", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "hostname": "smithi177", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "pid": "98215", 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "root": "/" 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:55.112 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_count": 0, 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_pinned_count": 0, 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "id": 4829, 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "inst": { 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "name": { 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "client", 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "num": 4829 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:55.113 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2771413729 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2771413729 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "inst_str": "client.4829 192.168.0.1:0/2771413729", 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "addr_str": "192.168.0.1:0/2771413729", 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "inode_count": 1, 2023-10-06T11:42:55.114 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_epoch": 17, 2023-10-06T11:42:55.115 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch": 32, 2023-10-06T11:42:55.115 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch_barrier": 0, 2023-10-06T11:42:55.115 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklisted": false, 2023-10-06T11:42:55.115 INFO:teuthology.orchestra.run.smithi177.stdout: "fs_name": "cephfs" 2023-10-06T11:42:55.115 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T11:42:55.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:55.122 DEBUG:teuthology.orchestra.run.smithi177:> stat --file-system '--printf=%T 2023-10-06T11:42:55.122 DEBUG:teuthology.orchestra.run.smithi177:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:55.136 INFO:teuthology.orchestra.run.smithi177.stdout:fuseblk 2023-10-06T11:42:55.137 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:55.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:55.137 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T11:42:55.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-06T11:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.337+0000 7f3cf1e96700 1 Processor -- start 2023-10-06T11:42:55.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.337+0000 7f3cf1e96700 1 -- start start 2023-10-06T11:42:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cf1e96700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:55.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cf1e96700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec159ba0 con 0x7f3cec0675c0 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cf1e96700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec0cd570 con 0x7f3cec0ce110 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cf1e96700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b0520 con 0x7f3cec0685e0 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3ceaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3cec0675c0 0x7f3cec0cda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36328/0 (socket says 172.21.15.16:36328) 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3ceaffd700 1 -- 172.21.15.16:0/287868038 learned_addr learned my addr 172.21.15.16:0/287868038 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/287868038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392129213 0 0) 0x7f3ce0001110 con 0x7f3cec0675c0 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd4003570 con 0x7f3cec0675c0 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802266492 0 0) 0x7f3cec0cd570 con 0x7f3cec0ce110 2023-10-06T11:42:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/287868038 >> v1:172.21.15.177:6789/0 conn(0x7f3cec0ce110 legacy=0x7f3cec06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.338+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/287868038 >> v1:172.21.15.16:6789/0 conn(0x7f3cec0675c0 legacy=0x7f3cec0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3ceb7fe700 1 -- 172.21.15.16:0/287868038 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0b2bf0 con 0x7f3cec0685e0 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/287868038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0b0c30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009910 tx=0x7f3cdc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24171cb3425e8d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc00b680 con 0x7f3cec0685e0 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc00b7e0 con 0x7f3cec0685e0 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cea7fc700 1 -- 172.21.15.16:0/287868038 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc00bba0 con 0x7f3cec0685e0 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cf1e96700 1 -- 172.21.15.16:0/287868038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 msgr2=0x7f3cec0b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/287868038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0b0c30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009910 tx=0x7f3cdc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cf1e96700 1 -- 172.21.15.16:0/287868038 shutdown_connections 2023-10-06T11:42:55.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/287868038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0b0c30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.339+0000 7f3cf1e96700 1 -- 172.21.15.16:0/287868038 >> 172.21.15.16:0/287868038 conn(0x7f3cec06d520 msgr2=0x7f3cec06d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.340+0000 7f3cf1e96700 1 -- 172.21.15.16:0/287868038 shutdown_connections 2023-10-06T11:42:55.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.340+0000 7f3cf1e96700 1 -- 172.21.15.16:0/287868038 wait complete. 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.341+0000 7f3cf1e96700 1 Processor -- start 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.341+0000 7f3cf1e96700 1 -- start start 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0675c0 0x7f3cec16b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:55.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0c3570 con 0x7f3cec0ce110 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec159d40 con 0x7f3cec0675c0 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec15a020 con 0x7f3cec0685e0 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47668/0 (socket says 172.21.15.16:47668) 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 -- 172.21.15.16:0/2343831149 learned_addr learned my addr 172.21.15.16:0/2343831149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0675c0 0x7f3cec16b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3ceaffd700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 -- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 msgr2=0x7f3cec0c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0c2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 -- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0675c0 msgr2=0x7f3cec16b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0675c0 0x7f3cec16b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 -- 172.21.15.16:0/2343831149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc009420 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3ceaffd700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cebfff700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c890 tx=0x7f3ce4001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4017070 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce4006ba0 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec15a300 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.342+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4012630 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.343+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec0c31b0 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.343+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec063e70 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.344+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ce4012a60 con 0x7f3cec0ce110 2023-10-06T11:42:55.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.344+0000 7f3cd37fe700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.344+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3ce4084430 con 0x7f3cec0ce110 2023-10-06T11:42:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.345+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.345+0000 7f3ceb7fe700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 0x7f3cd80693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0093f0 tx=0x7f3cdc017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:55.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.349+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ce4051540 con 0x7f3cec0ce110 2023-10-06T11:42:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.590+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3cec0b2ed0 con 0x7f3cec0ce110 2023-10-06T11:42:55.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.882+0000 7f3cd37fe700 1 -- 172.21.15.16:0/2343831149 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3ce4054b60 con 0x7f3cec0ce110 2023-10-06T11:42:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 msgr2=0x7f3cd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 0x7f3cd80693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0093f0 tx=0x7f3cdc017000 comp rx=0 tx=0).stop 2023-10-06T11:42:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 msgr2=0x7f3cec15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400c890 tx=0x7f3ce4001610 comp rx=0 tx=0).stop 2023-10-06T11:42:55.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 shutdown_connections 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cec0ce110 0x7f3cec15c830 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f3cd8066f30 0x7f3cd80693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cec0675c0 0x7f3cec16b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 --2- 172.21.15.16:0/2343831149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cec0685e0 0x7f3cec0c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 >> 172.21.15.16:0/2343831149 conn(0x7f3cec06d520 msgr2=0x7f3cec0ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 shutdown_connections 2023-10-06T11:42:55.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:55.885+0000 7f3cf1e96700 1 -- 172.21.15.16:0/2343831149 wait complete. 2023-10-06T11:42:55.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T11:42:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.021+0000 7fde2a8ee700 1 Processor -- start 2023-10-06T11:42:56.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.021+0000 7fde2a8ee700 1 -- start start 2023-10-06T11:42:56.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde2a8ee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fde24159800 0x7fde240b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde2a8ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde240b40b0 0x7fde240b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde2a8ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde240b0d50 con 0x7fde240b6360 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde2a8ee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde240b4a70 con 0x7fde240b40b0 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde2a8ee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde240b0140 con 0x7fde24159800 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde240b40b0 0x7fde240b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde240b40b0 0x7fde240b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 -- 172.21.15.16:0/2961077754 learned_addr learned my addr 172.21.15.16:0/2961077754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde23fff700 1 --2- 172.21.15.16:0/2961077754 >> v2:172.21.15.177:3300/0 conn(0x7fde24159800 0x7fde240b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 -- 172.21.15.16:0/2961077754 >> v2:172.21.15.177:3300/0 conn(0x7fde24159800 msgr2=0x7fde240b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:56.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 --2- 172.21.15.16:0/2961077754 >> v2:172.21.15.177:3300/0 conn(0x7fde24159800 0x7fde240b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 -- 172.21.15.16:0/2961077754 >> v1:172.21.15.177:6789/0 conn(0x7fde240b6360 legacy=0x7fde240b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 -- 172.21.15.16:0/2961077754 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240b0420 con 0x7fde240b40b0 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.022+0000 7fde28e8b700 1 --2- 172.21.15.16:0/2961077754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde240b4530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde24089ca0 tx=0x7fde1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617447f9ac7326d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde22ffd700 1 -- 172.21.15.16:0/2961077754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00e070 con 0x7fde240b40b0 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde22ffd700 1 -- 172.21.15.16:0/2961077754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c00ba20 con 0x7fde240b40b0 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde22ffd700 1 -- 172.21.15.16:0/2961077754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c0092e0 con 0x7fde240b40b0 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 -- 172.21.15.16:0/2961077754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 msgr2=0x7fde240b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/2961077754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde240b4530 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fde24089ca0 tx=0x7fde1c009840 comp rx=0 tx=0).stop 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 -- 172.21.15.16:0/2961077754 shutdown_connections 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/2961077754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde240b4530 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/2961077754 >> v2:172.21.15.177:3300/0 conn(0x7fde24159800 0x7fde240b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 -- 172.21.15.16:0/2961077754 >> 172.21.15.16:0/2961077754 conn(0x7fde2408a230 msgr2=0x7fde24099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 -- 172.21.15.16:0/2961077754 shutdown_connections 2023-10-06T11:42:56.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.023+0000 7fde2a8ee700 1 -- 172.21.15.16:0/2961077754 wait complete. 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.024+0000 7fde2a8ee700 1 Processor -- start 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 -- start start 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 0x7fde24156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde24159800 0x7fde24169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24157a50 con 0x7fde240b40b0 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde24155f00 con 0x7fde240b6360 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde2a8ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde24153270 con 0x7fde24159800 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde23fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47684/0 (socket says 172.21.15.16:47684) 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde23fff700 1 -- 172.21.15.16:0/1765410679 learned_addr learned my addr 172.21.15.16:0/1765410679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde28e8b700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde24159800 0x7fde24169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.025+0000 7fde237fe700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 0x7fde24156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 -- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde24159800 msgr2=0x7fde24169780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde24159800 0x7fde24169780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 -- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 msgr2=0x7fde24156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 0x7fde24156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 -- 172.21.15.16:0/1765410679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c009ae0 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde237fe700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 0x7fde24156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde23fff700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fde1400ecb0 tx=0x7fde140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400c8c0 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400ca40 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241571b0 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400ce00 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.026+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde24157770 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.027+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde24063e70 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.027+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde14013680 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.028+0000 7fde1bfff700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 0x7fde040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.028+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde14084910 con 0x7fde240b40b0 2023-10-06T11:42:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.028+0000 7fde237fe700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 0x7fde040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.028+0000 7fde237fe700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 0x7fde040693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde0c007fd0 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.032+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde14051a20 con 0x7fde240b40b0 2023-10-06T11:42:56.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.252+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fde241533b0 con 0x7fde04066f30 2023-10-06T11:42:56.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.253+0000 7fde1bfff700 1 -- 172.21.15.16:0/1765410679 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fde0c001d20 con 0x7fde04066f30 2023-10-06T11:42:56.256 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-10-06T11:42:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:42:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2023-10-06T11:42:56.256 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:42:56.256 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 msgr2=0x7fde040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 0x7fde040693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde0c007fd0 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 msgr2=0x7fde24156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fde1400ecb0 tx=0x7fde140091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 shutdown_connections 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde240b40b0 0x7fde24156410 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fde04066f30 0x7fde040693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde24159800 0x7fde24169780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 --2- 172.21.15.16:0/1765410679 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde240b6360 0x7fde24156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.255+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 >> 172.21.15.16:0/1765410679 conn(0x7fde2408a230 msgr2=0x7fde2406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.256+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 shutdown_connections 2023-10-06T11:42:56.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.256+0000 7fde2a8ee700 1 -- 172.21.15.16:0/1765410679 wait complete. 2023-10-06T11:42:56.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T11:42:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.389+0000 7f0c9d087700 1 Processor -- start 2023-10-06T11:42:56.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.389+0000 7f0c9d087700 1 -- start start 2023-10-06T11:42:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9d087700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9d087700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9d087700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c980cd900 con 0x7f0c980cd4a0 2023-10-06T11:42:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9d087700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c980cffc0 con 0x7f0c980ced70 2023-10-06T11:42:56.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9d087700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980cf3b0 con 0x7f0c981594a0 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c977fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:33144/0 (socket says 172.21.15.16:33144) 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c977fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:55102/0 (socket says 172.21.15.16:55102) 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c97fff700 1 -- 172.21.15.16:0/937276864 learned_addr learned my addr 172.21.15.16:0/937276864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.390+0000 7f0c9c886700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0c980cd4a0 0x7f0c980c54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:40556/0 (socket says 172.21.15.16:40556) 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c97fff700 1 -- 172.21.15.16:0/937276864 >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 msgr2=0x7f0c980ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c97fff700 1 --2- 172.21.15.16:0/937276864 >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c97fff700 1 -- 172.21.15.16:0/937276864 >> v1:172.21.15.177:6789/0 conn(0x7f0c980cd4a0 legacy=0x7f0c980c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c97fff700 1 -- 172.21.15.16:0/937276864 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c980cf690 con 0x7f0c981594a0 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c977fe700 1 --2- 172.21.15.16:0/937276864 >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:56.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c97fff700 1 --2- 172.21.15.16:0/937276864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009990 tx=0x7f0c88009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140b2530ff412960 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c96ffd700 1 -- 172.21.15.16:0/937276864 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88009e80 con 0x7f0c981594a0 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c96ffd700 1 -- 172.21.15.16:0/937276864 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88005c10 con 0x7f0c981594a0 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.391+0000 7f0c96ffd700 1 -- 172.21.15.16:0/937276864 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800b650 con 0x7f0c981594a0 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 -- 172.21.15.16:0/937276864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 msgr2=0x7f0c980ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 --2- 172.21.15.16:0/937276864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009990 tx=0x7f0c88009220 comp rx=0 tx=0).stop 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 -- 172.21.15.16:0/937276864 shutdown_connections 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 --2- 172.21.15.16:0/937276864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c981594a0 0x7f0c980ce830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 --2- 172.21.15.16:0/937276864 >> v2:172.21.15.177:3300/0 conn(0x7f0c980ced70 0x7f0c980ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 -- 172.21.15.16:0/937276864 >> 172.21.15.16:0/937276864 conn(0x7f0c98089f90 msgr2=0x7f0c980998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 -- 172.21.15.16:0/937276864 shutdown_connections 2023-10-06T11:42:56.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.392+0000 7f0c9d087700 1 -- 172.21.15.16:0/937276864 wait complete. 2023-10-06T11:42:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.393+0000 7f0c9d087700 1 Processor -- start 2023-10-06T11:42:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.393+0000 7f0c9d087700 1 -- start start 2023-10-06T11:42:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 0x7f0c9814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c981594a0 0x7f0c98148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9814e910 con 0x7f0c980ced70 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980c87a0 con 0x7f0c981594a0 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9d087700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981509d0 con 0x7f0c980cd4a0 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c97fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 0x7f0c9814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47698/0 (socket says 172.21.15.16:47698) 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 -- 172.21.15.16:0/2261439915 learned_addr learned my addr 172.21.15.16:0/2261439915 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c9c886700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c981594a0 0x7f0c98148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 -- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 msgr2=0x7f0c9814e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 0x7f0c9814e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 -- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c981594a0 msgr2=0x7f0c98148110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c981594a0 0x7f0c98148110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 -- 172.21.15.16:0/2261439915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c80009420 con 0x7f0c980ced70 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c97fff700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 0x7f0c9814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.394+0000 7f0c977fe700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000ec80 tx=0x7f0c800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.395+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000c920 con 0x7f0c980ced70 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.395+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000ca80 con 0x7f0c980ced70 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.395+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c88009480 con 0x7f0c980ced70 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.395+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000ce00 con 0x7f0c980ced70 2023-10-06T11:42:56.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.395+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c980c8150 con 0x7f0c980ced70 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.396+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98089530 con 0x7f0c980ced70 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.396+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c80013680 con 0x7f0c980ced70 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.396+0000 7f0c8ffff700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.396+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c80084da0 con 0x7f0c980ced70 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.397+0000 7f0c97fff700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.397+0000 7f0c97fff700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 0x7f0c780693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009260 tx=0x7f0c88016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.401+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c80051eb0 con 0x7f0c980ced70 2023-10-06T11:42:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.594+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0c980c88e0 con 0x7f0c980ced70 2023-10-06T11:42:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.594+0000 7f0c8ffff700 1 -- 172.21.15.16:0/2261439915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0c800170c0 con 0x7f0c980ced70 2023-10-06T11:42:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 msgr2=0x7f0c780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 0x7f0c780693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009260 tx=0x7f0c88016000 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 msgr2=0x7f0c980ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000ec80 tx=0x7f0c800091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 shutdown_connections 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f0c78066f30 0x7f0c780693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0c981594a0 0x7f0c98148110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0c980cd4a0 0x7f0c9814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 --2- 172.21.15.16:0/2261439915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c980ced70 0x7f0c980ca000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 >> 172.21.15.16:0/2261439915 conn(0x7f0c98089f90 msgr2=0x7f0c980b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 shutdown_connections 2023-10-06T11:42:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.596+0000 7f0c9d087700 1 -- 172.21.15.16:0/2261439915 wait complete. 2023-10-06T11:42:56.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000869203 2023-10-06T11:42:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.730+0000 7ff13711b700 1 Processor -- start 2023-10-06T11:42:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.730+0000 7ff13711b700 1 -- start start 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff13711b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff130159270 0x7ff13006acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff13711b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff130159640 0x7ff1300703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff13711b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13006a820 con 0x7ff13006e9b0 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff13711b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300abe40 con 0x7ff130159270 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff13711b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300ac120 con 0x7ff130159640 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff136119700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff130159270 0x7ff13006acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff136119700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff130159270 0x7ff13006acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47700/0 (socket says 172.21.15.16:47700) 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff136119700 1 -- 172.21.15.16:0/2395158290 learned_addr learned my addr 172.21.15.16:0/2395158290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.731+0000 7ff135918700 1 --2- 172.21.15.16:0/2395158290 >> v2:172.21.15.177:3300/0 conn(0x7ff130159640 0x7ff1300703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff136119700 1 -- 172.21.15.16:0/2395158290 >> v2:172.21.15.177:3300/0 conn(0x7ff130159640 msgr2=0x7ff1300703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff136119700 1 --2- 172.21.15.16:0/2395158290 >> v2:172.21.15.177:3300/0 conn(0x7ff130159640 0x7ff1300703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff136119700 1 -- 172.21.15.16:0/2395158290 >> v1:172.21.15.177:6789/0 conn(0x7ff13006e9b0 legacy=0x7ff13015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff136119700 1 -- 172.21.15.16:0/2395158290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13006a4b0 con 0x7ff130159270 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff136119700 1 --2- 172.21.15.16:0/2395158290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13006acb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff13006a2c0 tx=0x7ff120009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878475dff4a0583 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff135117700 1 -- 172.21.15.16:0/2395158290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12000e070 con 0x7ff130159270 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff135117700 1 -- 172.21.15.16:0/2395158290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12000ba20 con 0x7ff130159270 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff135117700 1 -- 172.21.15.16:0/2395158290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200092e0 con 0x7ff130159270 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff13711b700 1 -- 172.21.15.16:0/2395158290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 msgr2=0x7ff13006acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.732+0000 7ff13711b700 1 --2- 172.21.15.16:0/2395158290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13006acb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff13006a2c0 tx=0x7ff120009840 comp rx=0 tx=0).stop 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 -- 172.21.15.16:0/2395158290 shutdown_connections 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 --2- 172.21.15.16:0/2395158290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13006acb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 --2- 172.21.15.16:0/2395158290 >> v2:172.21.15.177:3300/0 conn(0x7ff130159640 0x7ff1300703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 -- 172.21.15.16:0/2395158290 >> 172.21.15.16:0/2395158290 conn(0x7ff130067cb0 msgr2=0x7ff1300680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 -- 172.21.15.16:0/2395158290 shutdown_connections 2023-10-06T11:42:56.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.733+0000 7ff13711b700 1 -- 172.21.15.16:0/2395158290 wait complete. 2023-10-06T11:42:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 Processor -- start 2023-10-06T11:42:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 -- start start 2023-10-06T11:42:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff13006e9b0 0x7ff130169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 0x7ff13006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130159be0 con 0x7ff130159270 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130159ec0 con 0x7ff130159640 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.734+0000 7ff13711b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13015a1a0 con 0x7ff13006e9b0 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47706/0 (socket says 172.21.15.16:47706) 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 -- 172.21.15.16:0/3962279830 learned_addr learned my addr 172.21.15.16:0/3962279830 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff136119700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff13006e9b0 0x7ff130169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff13691a700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 0x7ff13006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 -- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff13006e9b0 msgr2=0x7ff130169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff13006e9b0 0x7ff130169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 -- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 msgr2=0x7ff13006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 0x7ff13006d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 -- 172.21.15.16:0/3962279830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009ae0 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff13691a700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 0x7ff13006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff135918700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff12400ec80 tx=0x7ff1240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400c9d0 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12400cb30 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13015a480 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124013470 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.735+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13016ce00 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.736+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff130069550 con 0x7ff130159270 2023-10-06T11:42:56.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.737+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff124013820 con 0x7ff130159270 2023-10-06T11:42:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.737+0000 7ff12e7fc700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.737+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff124084a00 con 0x7ff130159270 2023-10-06T11:42:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.737+0000 7ff136119700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:56.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.738+0000 7ff136119700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 0x7ff11c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff13006a2c0 tx=0x7ff120018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.741+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff124051b10 con 0x7ff130159270 2023-10-06T11:42:56.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.962+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7ff13004bb40 con 0x7ff11c066f30 2023-10-06T11:42:56.970 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:56.968+0000 7fc9b6d43700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:56.970 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:56.968+0000 7fc9b6d43700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:56.970 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:56.968+0000 7fc9b6d43700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:56.970 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:56.968+0000 7fc9b6d43700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:56.970 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:56.968+0000 7fc9b6d43700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.983+0000 7ff12e7fc700 1 -- 172.21.15.16:0/3962279830 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff13004bb40 con 0x7ff11c066f30 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 msgr2=0x7ff11c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 0x7ff11c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff13006a2c0 tx=0x7ff120018000 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 msgr2=0x7ff13016c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff12400ec80 tx=0x7ff1240091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 shutdown_connections 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7ff11c066f30 0x7ff11c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff130159640 0x7ff13006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff13006e9b0 0x7ff130169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 --2- 172.21.15.16:0/3962279830 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff130159270 0x7ff13016c8c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 >> 172.21.15.16:0/3962279830 conn(0x7ff130067cb0 msgr2=0x7ff1300b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 shutdown_connections 2023-10-06T11:42:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:56.985+0000 7ff13711b700 1 -- 172.21.15.16:0/3962279830 wait complete. 2023-10-06T11:42:56.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000434953 --group_name group_0000000000869203 2023-10-06T11:42:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.121+0000 7f6cf8b86700 1 Processor -- start 2023-10-06T11:42:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.121+0000 7f6cf8b86700 1 -- start start 2023-10-06T11:42:57.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf8b86700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cf40afa80 0x7f6cf40b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf8b86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40b5f10 0x7f6cf40d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf8b86700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf40cedf0 con 0x7f6cf40af6b0 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf8b86700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf40d1560 con 0x7f6cf40afa80 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf8b86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40cd650 con 0x7f6cf40b5f10 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cf40afa80 0x7f6cf40b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cf40afa80 0x7f6cf40b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47708/0 (socket says 172.21.15.16:47708) 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf2ffd700 1 -- 172.21.15.16:0/170604689 learned_addr learned my addr 172.21.15.16:0/170604689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf3fff700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40b5f10 0x7f6cf40d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf2ffd700 1 -- 172.21.15.16:0/170604689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40b5f10 msgr2=0x7f6cf40d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.122+0000 7f6cf2ffd700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40b5f10 0x7f6cf40d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf2ffd700 1 -- 172.21.15.16:0/170604689 >> v1:172.21.15.177:6789/0 conn(0x7f6cf40af6b0 legacy=0x7f6cf415a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf2ffd700 1 -- 172.21.15.16:0/170604689 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40b6480 con 0x7f6cf40afa80 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf2ffd700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf40afa80 0x7f6cf40b33b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80099b0 tx=0x7f6ce8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911a53ae6a70dc53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf27fc700 1 -- 172.21.15.16:0/170604689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800b860 con 0x7f6cf40afa80 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf27fc700 1 -- 172.21.15.16:0/170604689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce800b9c0 con 0x7f6cf40afa80 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf27fc700 1 -- 172.21.15.16:0/170604689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800bd60 con 0x7f6cf40afa80 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 -- 172.21.15.16:0/170604689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf40afa80 msgr2=0x7f6cf40b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf40afa80 0x7f6cf40b33b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80099b0 tx=0x7f6ce8009250 comp rx=0 tx=0).stop 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 -- 172.21.15.16:0/170604689 shutdown_connections 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40b5f10 0x7f6cf40d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/170604689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf40afa80 0x7f6cf40b33b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 -- 172.21.15.16:0/170604689 >> 172.21.15.16:0/170604689 conn(0x7f6cf406cea0 msgr2=0x7f6cf406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 -- 172.21.15.16:0/170604689 shutdown_connections 2023-10-06T11:42:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.123+0000 7f6cf8b86700 1 -- 172.21.15.16:0/170604689 wait complete. 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 Processor -- start 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 -- start start 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40af6b0 0x7f6cf41472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6cf40b5f10 0x7f6cf416d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf416a250 con 0x7f6cf41569d0 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf4147800 con 0x7f6cf40b5f10 2023-10-06T11:42:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.125+0000 7f6cf8b86700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf416d770 con 0x7f6cf40af6b0 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47718/0 (socket says 172.21.15.16:47718) 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 -- 172.21.15.16:0/380017111 learned_addr learned my addr 172.21.15.16:0/380017111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf2ffd700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6cf40b5f10 0x7f6cf416d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf37fe700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40af6b0 0x7f6cf41472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 -- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40af6b0 msgr2=0x7f6cf41472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40af6b0 0x7f6cf41472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 -- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6cf40b5f10 msgr2=0x7f6cf416d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6cf40b5f10 0x7f6cf416d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 -- 172.21.15.16:0/380017111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce80094f0 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf3fff700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6cec007f20 tx=0x7f6cec009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00eb90 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec00ed10 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf4169c90 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec014670 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.126+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf416ca30 con 0x7f6cf41569d0 2023-10-06T11:42:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.127+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf404c380 con 0x7f6cf41569d0 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.128+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cec0149e0 con 0x7f6cf41569d0 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.128+0000 7f6cdb7fe700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 0x7f6ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.128+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cec084c90 con 0x7f6cf41569d0 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.128+0000 7f6cf37fe700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 0x7f6ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.129+0000 7f6cf37fe700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 0x7f6ce00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4003a80 tx=0x7f6ce4007ec0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.132+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cec051da0 con 0x7f6cf41569d0 2023-10-06T11:42:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.356+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434953", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cf40cd090 con 0x7f6ce0066f30 2023-10-06T11:42:57.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.383+0000 7f6cdb7fe700 1 -- 172.21.15.16:0/380017111 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6cf40cd090 con 0x7f6ce0066f30 2023-10-06T11:42:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 msgr2=0x7f6ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 0x7f6ce00693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4003a80 tx=0x7f6ce4007ec0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 msgr2=0x7f6cf41692d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6cec007f20 tx=0x7f6cec009860 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 shutdown_connections 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cf41569d0 0x7f6cf41692d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f6ce0066f30 0x7f6ce00693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6cf40af6b0 0x7f6cf41472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 --2- 172.21.15.16:0/380017111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6cf40b5f10 0x7f6cf416d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 >> 172.21.15.16:0/380017111 conn(0x7f6cf406cea0 msgr2=0x7f6cf40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 shutdown_connections 2023-10-06T11:42:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.385+0000 7f6cf8b86700 1 -- 172.21.15.16:0/380017111 wait complete. 2023-10-06T11:42:57.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000434954 --group_name group_0000000000869203 2023-10-06T11:42:57.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.521+0000 7f2300af4700 1 Processor -- start 2023-10-06T11:42:57.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.521+0000 7f2300af4700 1 -- start start 2023-10-06T11:42:57.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f2300af4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22fc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f2300af4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc0b2630 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f2300af4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22fc0b1a80 con 0x7f22fc159810 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f2300af4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0b31d0 con 0x7f22fc0ab040 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22fad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f22fc0b3fc0 0x7f22fc159be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36336/0 (socket says 172.21.15.16:36336) 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22fad9d700 1 -- 172.21.15.16:0/1007281139 learned_addr learned my addr 172.21.15.16:0/1007281139 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938450534 0 0) 0x7f22fc0b2630 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22f9d9b700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22fc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e8003570 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3716429288 0 0) 0x7f22e8003570 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.522+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 msgr2=0x7f22fc0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22fc0b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 >> v1:172.21.15.177:6789/0 conn(0x7f22fc159810 legacy=0x7f22fc0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0b1d00 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f22f40017f0 con 0x7f22fc0b3fc0 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 >> v1:172.21.15.16:6789/0 conn(0x7f22fc0b3fc0 legacy=0x7f22fc159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e8003a90 0x7f22e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e8004d50 0x7f22e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0b31d0 con 0x7f22e8003a90 2023-10-06T11:42:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e800bb70 con 0x7f22e8004d50 2023-10-06T11:42:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e800bd30 con 0x7f22fc0ab040 2023-10-06T11:42:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e8003a90 0x7f22e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f9d9b700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fad9d700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e8004d50 0x7f22e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 -- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 msgr2=0x7f22e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22fc0ab040 0x7f22e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 -- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e8004d50 msgr2=0x7f22e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22e8004d50 0x7f22e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 -- 172.21.15.16:0/1007281139 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e8003570 con 0x7f22e8003a90 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22fa59c700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e8003a90 0x7f22e8003ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0018b0 tx=0x7f22ec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b46e097e76c7aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec012420 con 0x7f22e8003a90 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec012580 con 0x7f22e8003a90 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.523+0000 7f22f959a700 1 -- 172.21.15.16:0/1007281139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec012920 con 0x7f22e8003a90 2023-10-06T11:42:57.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 -- 172.21.15.16:0/1007281139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e8003a90 msgr2=0x7f22e8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 --2- 172.21.15.16:0/1007281139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22e8003a90 0x7f22e8003ea0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0018b0 tx=0x7f22ec007d30 comp rx=0 tx=0).stop 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f22fad9d700 1 -- 172.21.15.16:0/1007281139 reap_dead start 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 -- 172.21.15.16:0/1007281139 shutdown_connections 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 -- 172.21.15.16:0/1007281139 >> 172.21.15.16:0/1007281139 conn(0x7f22fc08a1e0 msgr2=0x7f22fc099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 -- 172.21.15.16:0/1007281139 shutdown_connections 2023-10-06T11:42:57.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.524+0000 7f2300af4700 1 -- 172.21.15.16:0/1007281139 wait complete. 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.525+0000 7f2300af4700 1 Processor -- start 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.525+0000 7f2300af4700 1 -- start start 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e8003a90 0x7f22fc16a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22fc0ac7a0 0x7f22fc0acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0afcb0 con 0x7f22fc16c3b0 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc0ad150 con 0x7f22fc0ac7a0 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f2300af4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc16c820 con 0x7f22e8003a90 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47742/0 (socket says 172.21.15.16:47742) 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 -- 172.21.15.16:0/137147775 learned_addr learned my addr 172.21.15.16:0/137147775 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22fa59c700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e8003a90 0x7f22fc16a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22fad9d700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22fc0ac7a0 0x7f22fc0acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 -- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e8003a90 msgr2=0x7f22fc16a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e8003a90 0x7f22fc16a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 -- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22fc0ac7a0 msgr2=0x7f22fc0acc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22fc0ac7a0 0x7f22fc0acc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.526+0000 7f22f9d9b700 1 -- 172.21.15.16:0/137147775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc0af0a0 con 0x7f22fc16c3b0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f22f9d9b700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011d70 tx=0x7f22e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4009e20 con 0x7f22fc16c3b0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400fcd0 con 0x7f22fc16c3b0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22fc0af380 con 0x7f22fc16c3b0 2023-10-06T11:42:57.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4014650 con 0x7f22fc16c3b0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.527+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22fc0af940 con 0x7f22fc16c3b0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.528+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc16a650 con 0x7f22fc16c3b0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.528+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22e4014a80 con 0x7f22fc16c3b0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.528+0000 7f22f27fc700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.529+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f22e4018030 con 0x7f22fc16c3b0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.529+0000 7f22fa59c700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.529+0000 7f22fa59c700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 0x7f22dc0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22ec001690 tx=0x7f22ec015000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.533+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22fc16a650 con 0x7f22fc16c3b0 2023-10-06T11:42:57.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.758+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434954", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7f22fc16a650 con 0x7f22dc066f30 2023-10-06T11:42:57.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.785+0000 7f22f27fc700 1 -- 172.21.15.16:0/137147775 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22fc16a650 con 0x7f22dc066f30 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.787+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 msgr2=0x7f22dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.787+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 0x7f22dc0693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f22ec001690 tx=0x7f22ec015000 comp rx=0 tx=0).stop 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.787+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 msgr2=0x7f22fc0ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.787+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011d70 tx=0x7f22e40091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 shutdown_connections 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f22dc066f30 0x7f22dc0693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f22fc0ac7a0 0x7f22fc0acc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f22e8003a90 0x7f22fc16a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 --2- 172.21.15.16:0/137147775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22fc16c3b0 0x7f22fc0ae7e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 >> 172.21.15.16:0/137147775 conn(0x7f22fc08a1e0 msgr2=0x7f22fc0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 shutdown_connections 2023-10-06T11:42:57.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.788+0000 7f2300af4700 1 -- 172.21.15.16:0/137147775 wait complete. 2023-10-06T11:42:57.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000434955 --group_name group_0000000000869203 2023-10-06T11:42:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.925+0000 7fea5252e700 1 Processor -- start 2023-10-06T11:42:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.925+0000 7fea5252e700 1 -- start start 2023-10-06T11:42:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea5252e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea4c0b2a40 0x7fea4c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea5252e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cd900 0x7fea4c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea5252e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0cca70 con 0x7fea4c0cb3f0 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea5252e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0cf950 con 0x7fea4c0b2a40 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea5252e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0cbe60 con 0x7fea4c0cd900 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea4c0b2a40 0x7fea4c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea4c0cb3f0 0x7fea4c0cd1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4b7fe700 1 -- 172.21.15.16:0/2774367484 learned_addr learned my addr 172.21.15.16:0/2774367484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea50acb700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cd900 0x7fea4c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4affd700 1 -- 172.21.15.16:0/2774367484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950368937 0 0) 0x7fea4c0cca70 con 0x7fea4c0cb3f0 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4affd700 1 -- 172.21.15.16:0/2774367484 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003570 con 0x7fea4c0cb3f0 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 -- 172.21.15.16:0/2774367484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cd900 msgr2=0x7fea4c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:57.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cd900 0x7fea4c0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 -- 172.21.15.16:0/2774367484 >> v1:172.21.15.16:6789/0 conn(0x7fea4c0cb3f0 legacy=0x7fea4c0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 -- 172.21.15.16:0/2774367484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0cc140 con 0x7fea4c0b2a40 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.926+0000 7fea4bfff700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c0b2e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fea340098d0 tx=0x7fea34009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4146d0081ba834 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea4affd700 1 -- 172.21.15.16:0/2774367484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34005c10 con 0x7fea4c0b2a40 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea4affd700 1 -- 172.21.15.16:0/2774367484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea34005d70 con 0x7fea4c0b2a40 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea4affd700 1 -- 172.21.15.16:0/2774367484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34006280 con 0x7fea4c0b2a40 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 -- 172.21.15.16:0/2774367484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 msgr2=0x7fea4c0b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c0b2e10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fea340098d0 tx=0x7fea34009120 comp rx=0 tx=0).stop 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 -- 172.21.15.16:0/2774367484 shutdown_connections 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c0b2e10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 --2- 172.21.15.16:0/2774367484 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cd900 0x7fea4c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 -- 172.21.15.16:0/2774367484 >> 172.21.15.16:0/2774367484 conn(0x7fea4c08a230 msgr2=0x7fea4c099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 -- 172.21.15.16:0/2774367484 shutdown_connections 2023-10-06T11:42:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.927+0000 7fea5252e700 1 -- 172.21.15.16:0/2774367484 wait complete. 2023-10-06T11:42:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.928+0000 7fea5252e700 1 Processor -- start 2023-10-06T11:42:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 -- start start 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cb3f0 0x7fea4c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 0x7fea4c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c09a8d0 con 0x7fea4c0b2a40 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c09deb0 con 0x7fea4c0cd900 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea5252e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c09e190 con 0x7fea4c0cb3f0 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea4bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47756/0 (socket says 172.21.15.16:47756) 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.929+0000 7fea4bfff700 1 -- 172.21.15.16:0/2794228112 learned_addr learned my addr 172.21.15.16:0/2794228112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea50acb700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 0x7fea4c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4b7fe700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cb3f0 0x7fea4c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 -- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cb3f0 msgr2=0x7fea4c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cb3f0 0x7fea4c0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 -- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 msgr2=0x7fea4c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 0x7fea4c1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 -- 172.21.15.16:0/2794228112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340093c0 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea50acb700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 0x7fea4c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea4bfff700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fea34009f70 tx=0x7fea34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3401d800 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3401d960 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c09a560 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3401dd00 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.930+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0b5830 con 0x7fea4c0b2a40 2023-10-06T11:42:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.931+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c063e70 con 0x7fea4c0b2a40 2023-10-06T11:42:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.931+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea3401b690 con 0x7fea4c0b2a40 2023-10-06T11:42:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.932+0000 7fea43fff700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.932+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fea34086350 con 0x7fea4c0b2a40 2023-10-06T11:42:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.932+0000 7fea4b7fe700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.932+0000 7fea4b7fe700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 0x7fea2c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0c7ac0 tx=0x7fea3c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:57.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:57.936+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea34053460 con 0x7fea4c0b2a40 2023-10-06T11:42:58.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.161+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434955", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7fea4c09aa10 con 0x7fea2c066f30 2023-10-06T11:42:58.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.188+0000 7fea43fff700 1 -- 172.21.15.16:0/2794228112 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea3c001cf0 con 0x7fea2c066f30 2023-10-06T11:42:58.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 msgr2=0x7fea2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 0x7fea2c0693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fea4c0c7ac0 tx=0x7fea3c009040 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 msgr2=0x7fea4c09d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fea34009f70 tx=0x7fea34016000 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 shutdown_connections 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea4c0b2a40 0x7fea4c09d8b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea4c0cd900 0x7fea4c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 --2- 172.21.15.16:0/2794228112 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fea4c0cb3f0 0x7fea4c0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 >> 172.21.15.16:0/2794228112 conn(0x7fea4c08a230 msgr2=0x7fea4c06ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 shutdown_connections 2023-10-06T11:42:58.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.190+0000 7fea5252e700 1 -- 172.21.15.16:0/2794228112 wait complete. 2023-10-06T11:42:58.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000434953 group_0000000000869203 2023-10-06T11:42:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.446+0000 7f650eb7b700 1 Processor -- start 2023-10-06T11:42:58.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.446+0000 7f650eb7b700 1 -- start start 2023-10-06T11:42:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650eb7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65080886d0 0x7f650815b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650eb7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081575c0 0x7f650815b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650eb7b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65080c8b80 con 0x7f65081571f0 2023-10-06T11:42:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650eb7b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080c8dd0 con 0x7f65080886d0 2023-10-06T11:42:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650eb7b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080c90b0 con 0x7f65081575c0 2023-10-06T11:42:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650d378700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65080886d0 0x7f650815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650d378700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65080886d0 0x7f650815b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39370/0 (socket says 172.21.15.16:39370) 2023-10-06T11:42:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.447+0000 7f650d378700 1 -- 172.21.15.16:0/2910988098 learned_addr learned my addr 172.21.15.16:0/2910988098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650d378700 1 -- 172.21.15.16:0/2910988098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081575c0 msgr2=0x7f650815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650d378700 1 --2- 172.21.15.16:0/2910988098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081575c0 0x7f650815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650d378700 1 -- 172.21.15.16:0/2910988098 >> v1:172.21.15.177:6789/0 conn(0x7f65081571f0 legacy=0x7f6508087fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650d378700 1 -- 172.21.15.16:0/2910988098 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080c9390 con 0x7f65080886d0 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650d378700 1 --2- 172.21.15.16:0/2910988098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650815b330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64f80099b0 tx=0x7f64f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62aedcc601bcec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650cb77700 1 -- 172.21.15.16:0/2910988098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800b820 con 0x7f65080886d0 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650cb77700 1 -- 172.21.15.16:0/2910988098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f800b980 con 0x7f65080886d0 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.448+0000 7f650cb77700 1 -- 172.21.15.16:0/2910988098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f800bd20 con 0x7f65080886d0 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 -- 172.21.15.16:0/2910988098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 msgr2=0x7f650815b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2910988098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650815b330 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64f80099b0 tx=0x7f64f8009250 comp rx=0 tx=0).stop 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 -- 172.21.15.16:0/2910988098 shutdown_connections 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2910988098 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081575c0 0x7f650815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2910988098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650815b330 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 -- 172.21.15.16:0/2910988098 >> 172.21.15.16:0/2910988098 conn(0x7f6508097730 msgr2=0x7f650806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 -- 172.21.15.16:0/2910988098 shutdown_connections 2023-10-06T11:42:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.449+0000 7f650eb7b700 1 -- 172.21.15.16:0/2910988098 wait complete. 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.450+0000 7f650eb7b700 1 Processor -- start 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.450+0000 7f650eb7b700 1 -- start start 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 0x7f65080ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 0x7f65080adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650814ffd0 con 0x7f65080886d0 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080acad0 con 0x7f65081575c0 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65080abec0 con 0x7f65081571f0 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39382/0 (socket says 172.21.15.16:39382) 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 -- 172.21.15.16:0/2922976777 learned_addr learned my addr 172.21.15.16:0/2922976777 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650d378700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 0x7f65080ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650e37a700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 0x7f65080adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 -- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 msgr2=0x7f65080ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 0x7f65080ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 -- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 msgr2=0x7f65080adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 0x7f65080adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 -- 172.21.15.16:0/2922976777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f80094f0 con 0x7f65080886d0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650e37a700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 0x7f65080adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650d378700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 0x7f65080ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650db79700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f650400c5b0 tx=0x7f65040020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011590 con 0x7f65080886d0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65040116f0 con 0x7f65080886d0 2023-10-06T11:42:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65080ac1a0 con 0x7f65080886d0 2023-10-06T11:42:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.451+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011a90 con 0x7f65080886d0 2023-10-06T11:42:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.452+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65080ac760 con 0x7f65080886d0 2023-10-06T11:42:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.453+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65080acdb0 con 0x7f65080886d0 2023-10-06T11:42:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.453+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6504012420 con 0x7f65080886d0 2023-10-06T11:42:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.453+0000 7f64fdffb700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 0x7f64ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.453+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6504084120 con 0x7f65080886d0 2023-10-06T11:42:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.453+0000 7f650d378700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 0x7f64ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.454+0000 7f650d378700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 0x7f64ec0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f64f80099b0 tx=0x7f64f8016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.458+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6504051230 con 0x7f65080886d0 2023-10-06T11:42:58.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.681+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434953", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7f65080acc10 con 0x7f64ec066f30 2023-10-06T11:42:58.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.696+0000 7f64fdffb700 1 -- 172.21.15.16:0/2922976777 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f64f800bbd0 con 0x7f64ec066f30 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.698+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 msgr2=0x7f64ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.698+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 0x7f64ec0693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f64f80099b0 tx=0x7f64f8016000 comp rx=0 tx=0).stop 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.698+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 msgr2=0x7f650814fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.698+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f650400c5b0 tx=0x7f65040020e0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 shutdown_connections 2023-10-06T11:42:58.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65080886d0 0x7f650814fa90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f64ec066f30 0x7f64ec0693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65081575c0 0x7f65080adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 --2- 172.21.15.16:0/2922976777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65081571f0 0x7f65080ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 >> 172.21.15.16:0/2922976777 conn(0x7f6508097730 msgr2=0x7f65081507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 shutdown_connections 2023-10-06T11:42:58.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.699+0000 7f650eb7b700 1 -- 172.21.15.16:0/2922976777 wait complete. 2023-10-06T11:42:58.702 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.700+0000 7fc9b9d49700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.702 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.700+0000 7fc9b9d49700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.703 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.700+0000 7fc9b9d49700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.703 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.700+0000 7fc9b9d49700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.703 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.700+0000 7fc9b9d49700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000434954 group_0000000000869203 2023-10-06T11:42:58.716 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.713+0000 7fc9b8d47700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.716 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.713+0000 7fc9b8d47700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.716 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.713+0000 7fc9b8d47700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.716 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.713+0000 7fc9b8d47700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.716 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T11:42:58.713+0000 7fc9b8d47700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T11:42:58.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:58.758 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 Processor -- start 2023-10-06T11:42:58.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 -- start start 2023-10-06T11:42:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3406dcf0 con 0x7f2b340cb950 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340cd7f0 con 0x7f2b340cafe0 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.838+0000 7f2b3a0c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340cd290 con 0x7f2b34159bb0 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b32ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:49398/0 (socket says 172.21.15.16:49398) 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b32ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:39836/0 (socket says 172.21.15.16:39836) 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b33fff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2b340cb950 0x7f2b340c9380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:49342/0 (socket says 172.21.15.16:49342) 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 -- 172.21.15.16:0/3897799382 learned_addr learned my addr 172.21.15.16:0/3897799382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 -- 172.21.15.16:0/3897799382 >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 msgr2=0x7f2b340cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 --2- 172.21.15.16:0/3897799382 >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 -- 172.21.15.16:0/3897799382 >> v1:172.21.15.177:6789/0 conn(0x7f2b340cb950 legacy=0x7f2b340c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 -- 172.21.15.16:0/3897799382 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cd510 con 0x7f2b34159bb0 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b327fc700 1 -- 172.21.15.16:0/3897799382 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136626155 0 0) 0x7f2b3406dcf0 con 0x7f2b340cb950 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b32ffd700 1 --2- 172.21.15.16:0/3897799382 >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b337fe700 1 --2- 172.21.15.16:0/3897799382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009990 tx=0x7f2b24009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73f34060fe43fe38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.839+0000 7f2b327fc700 1 -- 172.21.15.16:0/3897799382 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24009e80 con 0x7f2b34159bb0 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b327fc700 1 -- 172.21.15.16:0/3897799382 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b24005c10 con 0x7f2b34159bb0 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b327fc700 1 -- 172.21.15.16:0/3897799382 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400b6e0 con 0x7f2b34159bb0 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/3897799382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 msgr2=0x7f2b340ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/3897799382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009990 tx=0x7f2b24009220 comp rx=0 tx=0).stop 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/3897799382 shutdown_connections 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/3897799382 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b34159bb0 0x7f2b340ccd50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/3897799382 >> v2:172.21.15.177:3300/0 conn(0x7f2b340cafe0 0x7f2b340cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/3897799382 >> 172.21.15.16:0/3897799382 conn(0x7f2b3406cbf0 msgr2=0x7f2b34087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/3897799382 shutdown_connections 2023-10-06T11:42:58.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.840+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/3897799382 wait complete. 2023-10-06T11:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.841+0000 7f2b3a0c1700 1 Processor -- start 2023-10-06T11:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.841+0000 7f2b3a0c1700 1 -- start start 2023-10-06T11:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340cafe0 0x7f2b340c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b340cb950 0x7f2b341564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c4560 con 0x7f2b34159bb0 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c4ed0 con 0x7f2b340cafe0 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b3a0c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c5100 con 0x7f2b340cb950 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39388/0 (socket says 172.21.15.16:39388) 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b33fff700 1 -- 172.21.15.16:0/2811564808 learned_addr learned my addr 172.21.15.16:0/2811564808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b32ffd700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b340cb950 0x7f2b341564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.842+0000 7f2b337fe700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340cafe0 0x7f2b340c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 -- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b340cb950 msgr2=0x7f2b341564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b340cb950 0x7f2b341564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 -- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340cafe0 msgr2=0x7f2b340c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:58.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340cafe0 0x7f2b340c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 -- 172.21.15.16:0/2811564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28009420 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b33fff700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00ccc0 tx=0x7f2b2c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c005df0 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c00a670 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b24009480 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00aa30 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.843+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340b21f0 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.844+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34063e70 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.844+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b2c0070f0 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.845+0000 7f2b1b7fe700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 0x7f2b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.845+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2b2c084640 con 0x7f2b34159bb0 2023-10-06T11:42:58.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.845+0000 7f2b337fe700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 0x7f2b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:58.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.845+0000 7f2b337fe700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 0x7f2b200693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009260 tx=0x7f2b24016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:58.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.849+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b2c0517d0 con 0x7f2b34159bb0 2023-10-06T11:42:58.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:58.895+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2b2c051ba0 con 0x7f2b34159bb0 2023-10-06T11:42:59.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.076+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434954", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b3406d6a0 con 0x7f2b20066f30 2023-10-06T11:42:59.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.097+0000 7f2b1b7fe700 1 -- 172.21.15.16:0/2811564808 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2b3406d6a0 con 0x7f2b20066f30 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 msgr2=0x7f2b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 0x7f2b200693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009260 tx=0x7f2b24016000 comp rx=0 tx=0).stop 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 msgr2=0x7f2b340b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00ccc0 tx=0x7f2b2c001a20 comp rx=0 tx=0).stop 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 shutdown_connections 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b34159bb0 0x7f2b340b3360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2b20066f30 0x7f2b200693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b340cafe0 0x7f2b340c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 --2- 172.21.15.16:0/2811564808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2b340cb950 0x7f2b341564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 >> 172.21.15.16:0/2811564808 conn(0x7f2b3406cbf0 msgr2=0x7f2b3406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 shutdown_connections 2023-10-06T11:42:59.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.099+0000 7f2b3a0c1700 1 -- 172.21.15.16:0/2811564808 wait complete. 2023-10-06T11:42:59.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000434955 group_0000000000869203 2023-10-06T11:42:59.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.276+0000 7efdd8b5d700 1 Processor -- start 2023-10-06T11:42:59.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.276+0000 7efdd8b5d700 1 -- start start 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd8b5d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdd40d04e0 0x7efdd40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd8b5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40b4850 0x7efdd4163230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd8b5d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd40b33b0 con 0x7efdd40b6030 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd8b5d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd40b64d0 con 0x7efdd40d04e0 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd8b5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd4159a70 con 0x7efdd40b4850 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdd40d04e0 0x7efdd40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efdd40b6030 0x7efdd40d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49594/0 (socket says 172.21.15.16:49594) 2023-10-06T11:42:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdd40d04e0 0x7efdd40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39400/0 (socket says 172.21.15.16:39400) 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd2ffd700 1 -- 172.21.15.16:0/1089939970 learned_addr learned my addr 172.21.15.16:0/1089939970 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd3fff700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40b4850 0x7efdd4163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 -- 172.21.15.16:0/1089939970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40b4850 msgr2=0x7efdd4163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40b4850 0x7efdd4163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 -- 172.21.15.16:0/1089939970 >> v1:172.21.15.16:6789/0 conn(0x7efdd40b6030 legacy=0x7efdd40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.277+0000 7efdd37fe700 1 -- 172.21.15.16:0/1089939970 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd40cf800 con 0x7efdd40d04e0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd37fe700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40d04e0 0x7efdd40ccff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e60 tx=0x7efdc4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700a4bada7b563e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd27fc700 1 -- 172.21.15.16:0/1089939970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400e070 con 0x7efdd40d04e0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd27fc700 1 -- 172.21.15.16:0/1089939970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc400bc30 con 0x7efdd40d04e0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd27fc700 1 -- 172.21.15.16:0/1089939970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4012430 con 0x7efdd40d04e0 2023-10-06T11:42:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 -- 172.21.15.16:0/1089939970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40d04e0 msgr2=0x7efdd40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40d04e0 0x7efdd40ccff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e60 tx=0x7efdc4009780 comp rx=0 tx=0).stop 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 -- 172.21.15.16:0/1089939970 shutdown_connections 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40d04e0 0x7efdd40ccff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/1089939970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40b4850 0x7efdd4163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 -- 172.21.15.16:0/1089939970 >> 172.21.15.16:0/1089939970 conn(0x7efdd406cea0 msgr2=0x7efdd406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 -- 172.21.15.16:0/1089939970 shutdown_connections 2023-10-06T11:42:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.278+0000 7efdd8b5d700 1 -- 172.21.15.16:0/1089939970 wait complete. 2023-10-06T11:42:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 Processor -- start 2023-10-06T11:42:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 -- start start 2023-10-06T11:42:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 0x7efdd41504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 0x7efdd4153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd4154c50 con 0x7efdd40b6030 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd4153f90 con 0x7efdd40b4850 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.280+0000 7efdd8b5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd41509f0 con 0x7efdd40d04e0 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39416/0 (socket says 172.21.15.16:39416) 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 -- 172.21.15.16:0/3361669141 learned_addr learned my addr 172.21.15.16:0/3361669141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 -- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 msgr2=0x7efdd4153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd3fff700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 0x7efdd4153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd37fe700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 0x7efdd41504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 0x7efdd4153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 -- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 msgr2=0x7efdd41504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 0x7efdd41504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 -- 172.21.15.16:0/3361669141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc4009a20 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd3fff700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 0x7efdd4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd37fe700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 0x7efdd41504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd2ffd700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c7c0 tx=0x7efdc8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8005b30 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc8005cb0 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd40b1590 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc8004a80 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.281+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd41548e0 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.282+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd4154460 con 0x7efdd40b6030 2023-10-06T11:42:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.283+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7efdc80070f0 con 0x7efdd40b6030 2023-10-06T11:42:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.283+0000 7efdbb7fe700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 0x7efdc0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.283+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efdc8083b30 con 0x7efdd40b6030 2023-10-06T11:42:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.283+0000 7efdd37fe700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 0x7efdc0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.284+0000 7efdd37fe700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 0x7efdc0069470 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e60 tx=0x7efdc4019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.288+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdd4154460 con 0x7efdd40b6030 2023-10-06T11:42:59.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.511+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000434955", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7efdd4154460 con 0x7efdc0066fc0 2023-10-06T11:42:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.530+0000 7efdbb7fe700 1 -- 172.21.15.16:0/3361669141 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efdd4154460 con 0x7efdc0066fc0 2023-10-06T11:42:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 msgr2=0x7efdc0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 0x7efdc0069470 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009e60 tx=0x7efdc4019000 comp rx=0 tx=0).stop 2023-10-06T11:42:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 msgr2=0x7efdd4153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdc800c7c0 tx=0x7efdc8007f70 comp rx=0 tx=0).stop 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 shutdown_connections 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efdc0066fc0 0x7efdc0069470 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efdd40d04e0 0x7efdd4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdd40b4850 0x7efdd41504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 --2- 172.21.15.16:0/3361669141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdd40b6030 0x7efdd4153510 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 >> 172.21.15.16:0/3361669141 conn(0x7efdd406cea0 msgr2=0x7efdd415ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 shutdown_connections 2023-10-06T11:42:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.532+0000 7efdd8b5d700 1 -- 172.21.15.16:0/3361669141 wait complete. 2023-10-06T11:42:59.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs group_0000000000869203 2023-10-06T11:42:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 Processor -- start 2023-10-06T11:42:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 -- start start 2023-10-06T11:42:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 0x7f7e8c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8c0cead0 con 0x7f7e8c159b10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8c0affc0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.668+0000 7f7e914d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c0cdec0 con 0x7f7e8c159740 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e8c159b10 0x7f7e8c0cef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49598/0 (socket says 172.21.15.16:49598) 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8b7fe700 1 -- 172.21.15.16:0/682806463 learned_addr learned my addr 172.21.15.16:0/682806463 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8affd700 1 --2- 172.21.15.16:0/682806463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 0x7f7e8c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903188374 0 0) 0x7f7e8c0cead0 con 0x7f7e8c159b10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e78003570 con 0x7f7e8c159b10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645456572 0 0) 0x7f7e8c0affc0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e8c0cead0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8affd700 1 -- 172.21.15.16:0/682806463 >> v1:172.21.15.177:6789/0 conn(0x7f7e8c0cda10 legacy=0x7f7e8c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8affd700 1 -- 172.21.15.16:0/682806463 >> v1:172.21.15.16:6789/0 conn(0x7f7e8c159b10 legacy=0x7f7e8c0cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8affd700 1 -- 172.21.15.16:0/682806463 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0ce1a0 con 0x7f7e8c159740 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e8affd700 1 --2- 172.21.15.16:0/682806463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 0x7f7e8c0b3990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009910 tx=0x7f7e7c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb99f24ca18090f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.669+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011630 con 0x7f7e8c159740 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c011790 con 0x7f7e8c159740 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e89ffb700 1 -- 172.21.15.16:0/682806463 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c011b30 con 0x7f7e8c159740 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 -- 172.21.15.16:0/682806463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 msgr2=0x7f7e8c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 --2- 172.21.15.16:0/682806463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 0x7f7e8c0b3990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009910 tx=0x7f7e7c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 -- 172.21.15.16:0/682806463 shutdown_connections 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 --2- 172.21.15.16:0/682806463 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159740 0x7f7e8c0b3990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 -- 172.21.15.16:0/682806463 >> 172.21.15.16:0/682806463 conn(0x7f7e8c099a40 msgr2=0x7f7e8c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 -- 172.21.15.16:0/682806463 shutdown_connections 2023-10-06T11:42:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.670+0000 7f7e914d4700 1 -- 172.21.15.16:0/682806463 wait complete. 2023-10-06T11:42:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.671+0000 7f7e914d4700 1 Processor -- start 2023-10-06T11:42:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.671+0000 7f7e914d4700 1 -- start start 2023-10-06T11:42:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e8c159740 0x7f7e8c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159b10 0x7f7e8c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c09e3d0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c09e0f0 con 0x7f7e8c159740 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e914d4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c15b6c0 con 0x7f7e8c159b10 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39432/0 (socket says 172.21.15.16:39432) 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 -- 172.21.15.16:0/2825762832 learned_addr learned my addr 172.21.15.16:0/2825762832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8b7fe700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159b10 0x7f7e8c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8a7fc700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e8c159740 0x7f7e8c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 -- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159b10 msgr2=0x7f7e8c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159b10 0x7f7e8c0b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 -- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e8c159740 msgr2=0x7f7e8c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e8c159740 0x7f7e8c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.672+0000 7f7e8affd700 1 -- 172.21.15.16:0/2825762832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c009420 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e8affd700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009f00 tx=0x7f7e7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c01f070 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c000bc0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8c15b9a0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c025420 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.673+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e8c156ce0 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.674+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8c15be40 con 0x7f7e8c0cda10 2023-10-06T11:42:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.674+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7e7c023070 con 0x7f7e8c0cda10 2023-10-06T11:42:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.675+0000 7f7e82ffd700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 0x7f7e6c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:42:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.675+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7e7c085a70 con 0x7f7e8c0cda10 2023-10-06T11:42:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.675+0000 7f7e8a7fc700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 0x7f7e6c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:42:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.675+0000 7f7e8a7fc700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 0x7f7e6c069470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e740037e0 tx=0x7f7e74009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:42:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.679+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e7c052b90 con 0x7f7e8c0cda10 2023-10-06T11:42:59.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.903+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "group_0000000000869203", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e8c04c380 con 0x7f7e6c066fc0 2023-10-06T11:42:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.908+0000 7f7e82ffd700 1 -- 172.21.15.16:0/2825762832 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7e8c04c380 con 0x7f7e6c066fc0 2023-10-06T11:42:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 msgr2=0x7f7e6c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 0x7f7e6c069470 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7e740037e0 tx=0x7f7e74009040 comp rx=0 tx=0).stop 2023-10-06T11:42:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 msgr2=0x7f7e8c09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:42:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009f00 tx=0x7f7e7c017000 comp rx=0 tx=0).stop 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 shutdown_connections 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e8c0cda10 0x7f7e8c09a3b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7e6c066fc0 0x7f7e6c069470 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7e8c159b10 0x7f7e8c0b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 --2- 172.21.15.16:0/2825762832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e8c159740 0x7f7e8c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 >> 172.21.15.16:0/2825762832 conn(0x7f7e8c099a40 msgr2=0x7f7e8c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 shutdown_connections 2023-10-06T11:42:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:42:59.910+0000 7f7e914d4700 1 -- 172.21.15.16:0/2825762832 wait complete. 2023-10-06T11:42:59.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T11:42:59.924 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-10-06T11:42:59.960 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-10-06T11:42:59.960 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T11:42:59.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:43:00.041 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T11:43:00.092 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T11:43:00.092 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-10-06T11:43:00.092 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:43:00.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T11:43:00.125 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[103256]: fuse finished with error 0 and tester_r 0 2023-10-06T11:43:00.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:00.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:00.126 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T11:43:00.187 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi177.front.sepia.ceph.com... 2023-10-06T11:43:00.187 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T11:43:00.188 DEBUG:teuthology.orchestra.run.smithi177:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T11:43:00.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[98215]: fuse finished with error 0 and tester_r 0 2023-10-06T11:43:00.221 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:00.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:00.221 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T11:43:00.385 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:00.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:00.386 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T11:43:00.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:00.401 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T11:43:00.402 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:00.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:00.402 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T11:43:00.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:00.441 INFO:teuthology.orchestra.run.smithi177.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T11:43:00.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:43:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.558+0000 7f261b537700 1 Processor -- start 2023-10-06T11:43:00.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 -- start start 2023-10-06T11:43:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 0x7f2614159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26140ad5b0 0x7f26140ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140c9410 con 0x7f2614159900 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140ca250 con 0x7f26140ad5b0 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.559+0000 7f261b537700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c9640 con 0x7f261406cbb0 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26140ad5b0 0x7f26140ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f26192d3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2614159900 0x7f26140acea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49602/0 (socket says 172.21.15.16:49602) 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f26140ad5b0 0x7f26140ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39442/0 (socket says 172.21.15.16:39442) 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f26192d3700 1 -- 172.21.15.16:0/2388403349 learned_addr learned my addr 172.21.15.16:0/2388403349 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2618ad2700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 0x7f2614159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 -- 172.21.15.16:0/2388403349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 msgr2=0x7f2614159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:00.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 0x7f2614159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 -- 172.21.15.16:0/2388403349 >> v1:172.21.15.16:6789/0 conn(0x7f2614159900 legacy=0x7f26140acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2613fff700 1 -- 172.21.15.16:0/2388403349 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824470669 0 0) 0x7f26140c9410 con 0x7f2614159900 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 -- 172.21.15.16:0/2388403349 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140c9920 con 0x7f26140ad5b0 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2618ad2700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 0x7f2614159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2619ad4700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26140ad5b0 0x7f26140ca6e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098e0 tx=0x7f260c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc50a8c6f5c58e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2613fff700 1 -- 172.21.15.16:0/2388403349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c009e80 con 0x7f26140ad5b0 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2613fff700 1 -- 172.21.15.16:0/2388403349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00b860 con 0x7f26140ad5b0 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.560+0000 7f2613fff700 1 -- 172.21.15.16:0/2388403349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00bbe0 con 0x7f26140ad5b0 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 -- 172.21.15.16:0/2388403349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26140ad5b0 msgr2=0x7f26140ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26140ad5b0 0x7f26140ca6e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098e0 tx=0x7f260c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 -- 172.21.15.16:0/2388403349 shutdown_connections 2023-10-06T11:43:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26140ad5b0 0x7f26140ca6e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 --2- 172.21.15.16:0/2388403349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f261406cbb0 0x7f2614159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 -- 172.21.15.16:0/2388403349 >> 172.21.15.16:0/2388403349 conn(0x7f261406f040 msgr2=0x7f261406c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 -- 172.21.15.16:0/2388403349 shutdown_connections 2023-10-06T11:43:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.561+0000 7f261b537700 1 -- 172.21.15.16:0/2388403349 wait complete. 2023-10-06T11:43:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.562+0000 7f261b537700 1 Processor -- start 2023-10-06T11:43:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.562+0000 7f261b537700 1 -- start start 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 0x7f26140c1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 0x7f26140abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614169b50 con 0x7f261406cbb0 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26141590f0 con 0x7f26140ad5b0 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f261b537700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2614155830 con 0x7f2614159900 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39448/0 (socket says 172.21.15.16:39448) 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 -- 172.21.15.16:0/3835460408 learned_addr learned my addr 172.21.15.16:0/3835460408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f2619ad4700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 0x7f26140abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f2618ad2700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 0x7f26140c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 -- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 msgr2=0x7f26140abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 0x7f26140abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 -- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 msgr2=0x7f26140c1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 0x7f26140c1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f26192d3700 1 -- 172.21.15.16:0/3835460408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c009420 con 0x7f261406cbb0 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f2619ad4700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 0x7f26140abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.563+0000 7f2618ad2700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 0x7f26140c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f26192d3700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f260400cde0 tx=0x7f2604001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400ab60 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400acc0 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2614157590 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604007300 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.564+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2614157b50 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.565+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261404c380 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.565+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f26040076f0 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.565+0000 7f26117fa700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 0x7f2600069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.566+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2604084cb0 con 0x7f261406cbb0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.566+0000 7f2618ad2700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 0x7f2600069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.566+0000 7f2618ad2700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 0x7f2600069470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2608006f00 tx=0x7f2608008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.570+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2604051e50 con 0x7f261406cbb0 2023-10-06T11:43:00.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.813+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26140aba70 con 0x7f261406cbb0 2023-10-06T11:43:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.814+0000 7f26117fa700 1 -- 172.21.15.16:0/3835460408 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f2604055470 con 0x7f261406cbb0 2023-10-06T11:43:00.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:00.816 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:43:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 msgr2=0x7f2600069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 0x7f2600069470 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2608006f00 tx=0x7f2608008040 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 msgr2=0x7f26141552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f260400cde0 tx=0x7f2604001ae0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 shutdown_connections 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261406cbb0 0x7f26141552f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2600066fc0 0x7f2600069470 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2614159900 0x7f26140abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 --2- 172.21.15.16:0/3835460408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26140ad5b0 0x7f26140c1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 >> 172.21.15.16:0/3835460408 conn(0x7f261406f040 msgr2=0x7f26140cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:00.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 shutdown_connections 2023-10-06T11:43:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.816+0000 7f261b537700 1 -- 172.21.15.16:0/3835460408 wait complete. 2023-10-06T11:43:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-10-06T11:43:00.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.948+0000 7f8a81768700 1 Processor -- start 2023-10-06T11:43:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.948+0000 7f8a81768700 1 -- start start 2023-10-06T11:43:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a81768700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a7c15cc20 0x7f8a7c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a81768700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0cdb10 con 0x7f8a7c0cd6a0 2023-10-06T11:43:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a81768700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0ccb20 con 0x7f8a7c0cf450 2023-10-06T11:43:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a81768700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c0b3110 con 0x7f8a7c15cc20 2023-10-06T11:43:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a7affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a7c15cc20 0x7f8a7c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a7a7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a7c0cd6a0 0x7f8a7c0ced40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49618/0 (socket says 172.21.15.16:49618) 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a7affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a7c15cc20 0x7f8a7c0cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39458/0 (socket says 172.21.15.16:39458) 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.949+0000 7f8a7a7fc700 1 -- 172.21.15.16:0/2556588406 learned_addr learned my addr 172.21.15.16:0/2556588406 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a7affd700 1 -- 172.21.15.16:0/2556588406 >> v1:172.21.15.177:6789/0 conn(0x7f8a7c0cf450 legacy=0x7f8a7c0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a7affd700 1 -- 172.21.15.16:0/2556588406 >> v1:172.21.15.16:6789/0 conn(0x7f8a7c0cd6a0 legacy=0x7f8a7c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a7affd700 1 -- 172.21.15.16:0/2556588406 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0cf880 con 0x7f8a7c15cc20 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a79ffb700 1 -- 172.21.15.16:0/2556588406 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955796498 0 0) 0x7f8a7c0cdb10 con 0x7f8a7c0cd6a0 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a7affd700 1 --2- 172.21.15.16:0/2556588406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c0cd160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e60 tx=0x7f8a6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee49b1d497853bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a79ffb700 1 -- 172.21.15.16:0/2556588406 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c00e070 con 0x7f8a7c15cc20 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a79ffb700 1 -- 172.21.15.16:0/2556588406 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c00bc30 con 0x7f8a7c15cc20 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.950+0000 7f8a79ffb700 1 -- 172.21.15.16:0/2556588406 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c012430 con 0x7f8a7c15cc20 2023-10-06T11:43:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 -- 172.21.15.16:0/2556588406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 msgr2=0x7f8a7c0cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 --2- 172.21.15.16:0/2556588406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c0cd160 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e60 tx=0x7f8a6c009780 comp rx=0 tx=0).stop 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 -- 172.21.15.16:0/2556588406 shutdown_connections 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 --2- 172.21.15.16:0/2556588406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c0cd160 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 -- 172.21.15.16:0/2556588406 >> 172.21.15.16:0/2556588406 conn(0x7f8a7c099b40 msgr2=0x7f8a7c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 -- 172.21.15.16:0/2556588406 shutdown_connections 2023-10-06T11:43:00.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.951+0000 7f8a81768700 1 -- 172.21.15.16:0/2556588406 wait complete. 2023-10-06T11:43:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.952+0000 7f8a81768700 1 Processor -- start 2023-10-06T11:43:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.952+0000 7f8a81768700 1 -- start start 2023-10-06T11:43:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a7c0cd6a0 0x7f8a7c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a7c0cf450 0x7f8a7c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c09e160 con 0x7f8a7c15cc20 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0c8700 con 0x7f8a7c0cf450 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a81768700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0c7810 con 0x7f8a7c0cd6a0 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39462/0 (socket says 172.21.15.16:39462) 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 -- 172.21.15.16:0/207032770 learned_addr learned my addr 172.21.15.16:0/207032770 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7a7fc700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a7c0cf450 0x7f8a7c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7affd700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a7c0cd6a0 0x7f8a7c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 -- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a7c0cd6a0 msgr2=0x7f8a7c157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a7c0cd6a0 0x7f8a7c157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 -- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a7c0cf450 msgr2=0x7f8a7c0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a7c0cf450 0x7f8a7c0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 -- 172.21.15.16:0/207032770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009a20 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.953+0000 7f8a7b7fe700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400c560 tx=0x7f8a74007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.954+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74001120 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.954+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74011590 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.954+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0c7af0 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.954+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74011950 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.954+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0c80b0 con 0x7f8a7c15cc20 2023-10-06T11:43:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.955+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c04c380 con 0x7f8a7c15cc20 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.955+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8a74011d00 con 0x7f8a7c15cc20 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.955+0000 7f8a72ffd700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 0x7f8a5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.956+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a74083d40 con 0x7f8a7c15cc20 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.956+0000 7f8a7affd700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 0x7f8a5c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.956+0000 7f8a7affd700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 0x7f8a5c069590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0099f0 tx=0x7f8a6c019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:00.960+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a74050b30 con 0x7f8a7c15cc20 2023-10-06T11:43:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.158+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a7c0cbf10 con 0x7f8a7c15cc20 2023-10-06T11:43:01.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.159+0000 7f8a72ffd700 1 -- 172.21.15.16:0/207032770 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f8a74017070 con 0x7f8a7c15cc20 2023-10-06T11:43:01.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:01.162 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:46.714204+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 msgr2=0x7f8a5c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 0x7f8a5c069590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0099f0 tx=0x7f8a6c019000 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 msgr2=0x7f8a7c169e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400c560 tx=0x7f8a74007ef0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 shutdown_connections 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a7c15cc20 0x7f8a7c169e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f8a5c0670e0 0x7f8a5c069590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a7c0cd6a0 0x7f8a7c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 --2- 172.21.15.16:0/207032770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a7c0cf450 0x7f8a7c0b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 >> 172.21.15.16:0/207032770 conn(0x7f8a7c099b40 msgr2=0x7f8a7c0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 shutdown_connections 2023-10-06T11:43:01.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.161+0000 7f8a81768700 1 -- 172.21.15.16:0/207032770 wait complete. 2023-10-06T11:43:01.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T11:42:43.414603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T11:42:44.091338+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T11:43:01.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:43:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.295+0000 7f9ca3ef9700 1 Processor -- start 2023-10-06T11:43:01.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.295+0000 7f9ca3ef9700 1 -- start start 2023-10-06T11:43:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c9c159740 0x7f9c9c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 0x7f9c9c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159b10 0x7f9c9c0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c0cead0 con 0x7f9c9c159740 2023-10-06T11:43:01.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9c0affc0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca3ef9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0cdec0 con 0x7f9c9c159b10 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c9c159740 0x7f9c9c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c9c159740 0x7f9c9c0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39472/0 (socket says 172.21.15.16:39472) 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 -- 172.21.15.16:0/3489390584 learned_addr learned my addr 172.21.15.16:0/3489390584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca2496700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159b10 0x7f9c9c0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1494700 1 --2- 172.21.15.16:0/3489390584 >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 0x7f9c9c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 -- 172.21.15.16:0/3489390584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159b10 msgr2=0x7f9c9c0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159b10 0x7f9c9c0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 -- 172.21.15.16:0/3489390584 >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 msgr2=0x7f9c9c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/3489390584 >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 0x7f9c9c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 -- 172.21.15.16:0/3489390584 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9c0ce1a0 con 0x7f9c9c159740 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1494700 1 --2- 172.21.15.16:0/3489390584 >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 0x7f9c9c0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c159740 0x7f9c9c0b3990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06bd00 tx=0x7f9c8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7121d9eb067b5f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.296+0000 7f9ca0c93700 1 -- 172.21.15.16:0/3489390584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00e070 con 0x7f9c9c159740 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca0c93700 1 -- 172.21.15.16:0/3489390584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c00ba20 con 0x7f9c9c159740 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca0c93700 1 -- 172.21.15.16:0/3489390584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0092e0 con 0x7f9c9c159740 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/3489390584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c159740 msgr2=0x7f9c9c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c159740 0x7f9c9c0b3990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06bd00 tx=0x7f9c8c009840 comp rx=0 tx=0).stop 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/3489390584 shutdown_connections 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159b10 0x7f9c9c0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/3489390584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c159740 0x7f9c9c0b3990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/3489390584 >> v2:172.21.15.177:3300/0 conn(0x7f9c9c0cda10 0x7f9c9c0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/3489390584 >> 172.21.15.16:0/3489390584 conn(0x7f9c9c099a40 msgr2=0x7f9c9c06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/3489390584 shutdown_connections 2023-10-06T11:43:01.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.297+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/3489390584 wait complete. 2023-10-06T11:43:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.298+0000 7f9ca3ef9700 1 Processor -- start 2023-10-06T11:43:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.298+0000 7f9ca3ef9700 1 -- start start 2023-10-06T11:43:01.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159740 0x7f9c9c169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 0x7f9c9c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c0ac7c0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c169d90 con 0x7f9c9c159b10 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca3ef9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9c15d210 con 0x7f9c9c159740 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39482/0 (socket says 172.21.15.16:39482) 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 -- 172.21.15.16:0/2048706332 learned_addr learned my addr 172.21.15.16:0/2048706332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1494700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159740 0x7f9c9c169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca2496700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 0x7f9c9c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 -- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159740 msgr2=0x7f9c9c169850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159740 0x7f9c9c169850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 -- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 msgr2=0x7f9c9c15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 0x7f9c9c15ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca1c95700 1 -- 172.21.15.16:0/2048706332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c009ae0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.299+0000 7f9ca2496700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 0x7f9c9c15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9ca1c95700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06d350 tx=0x7f9c8c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c0032f0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8c008b80 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9c15a130 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c026460 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.300+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9c15a9d0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.301+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9c04c380 con 0x7f9c9c0cda10 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.301+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9c8c003040 con 0x7f9c9c0cda10 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.302+0000 7f9c99ffb700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 0x7f9c88069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.302+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9c8c0868b0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.302+0000 7f9ca1494700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 0x7f9c88069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.302+0000 7f9ca1494700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 0x7f9c88069590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006f00 tx=0x7f9c90008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.306+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c8c053630 con 0x7f9c9c0cda10 2023-10-06T11:43:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.500+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9c9c15a2c0 con 0x7f9c9c0cda10 2023-10-06T11:43:01.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.500+0000 7f9c99ffb700 1 -- 172.21.15.16:0/2048706332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v32) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9c8c056c50 con 0x7f9c9c0cda10 2023-10-06T11:43:01.502 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 msgr2=0x7f9c88069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 0x7f9c88069590 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006f00 tx=0x7f9c90008040 comp rx=0 tx=0).stop 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 msgr2=0x7f9c9c0ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c06d350 tx=0x7f9c8c018000 comp rx=0 tx=0).stop 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 shutdown_connections 2023-10-06T11:43:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c9c0cda10 0x7f9c9c0ac120 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9c880670e0 0x7f9c88069590 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c9c159b10 0x7f9c9c15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 --2- 172.21.15.16:0/2048706332 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c9c159740 0x7f9c9c169850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 >> 172.21.15.16:0/2048706332 conn(0x7f9c9c099a40 msgr2=0x7f9c9c0cd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 shutdown_connections 2023-10-06T11:43:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.502+0000 7f9ca3ef9700 1 -- 172.21.15.16:0/2048706332 wait complete. 2023-10-06T11:43:01.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:43:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.637+0000 7fc23e3a6700 1 Processor -- start 2023-10-06T11:43:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.637+0000 7fc23e3a6700 1 -- start start 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23e3a6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc238153cd0 0x7fc2380b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23e3a6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc2380b14e0 0x7fc2380b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23e3a6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23815e070 con 0x7fc238069240 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23e3a6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2380b1e70 con 0x7fc238153cd0 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23e3a6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23815d4a0 con 0x7fc2380b14e0 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc238153cd0 0x7fc2380b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc238153cd0 0x7fc2380b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39492/0 (socket says 172.21.15.16:39492) 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 -- 172.21.15.16:0/2515074534 learned_addr learned my addr 172.21.15.16:0/2515074534 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23cba3700 1 --2- 172.21.15.16:0/2515074534 >> v2:172.21.15.177:3300/0 conn(0x7fc2380b14e0 0x7fc2380b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 -- 172.21.15.16:0/2515074534 >> v2:172.21.15.177:3300/0 conn(0x7fc2380b14e0 msgr2=0x7fc2380b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 --2- 172.21.15.16:0/2515074534 >> v2:172.21.15.177:3300/0 conn(0x7fc2380b14e0 0x7fc2380b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 -- 172.21.15.16:0/2515074534 >> v1:172.21.15.177:6789/0 conn(0x7fc238069240 legacy=0x7fc2380696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 -- 172.21.15.16:0/2515074534 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23815d740 con 0x7fc238153cd0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.638+0000 7fc23d3a4700 1 --2- 172.21.15.16:0/2515074534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc2380b3bd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc2380b32c0 tx=0x7fc228009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf8314416f6b6d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc237fff700 1 -- 172.21.15.16:0/2515074534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800e070 con 0x7fc238153cd0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc237fff700 1 -- 172.21.15.16:0/2515074534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22800ba20 con 0x7fc238153cd0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc237fff700 1 -- 172.21.15.16:0/2515074534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280092e0 con 0x7fc238153cd0 2023-10-06T11:43:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 -- 172.21.15.16:0/2515074534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 msgr2=0x7fc2380b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/2515074534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc2380b3bd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc2380b32c0 tx=0x7fc228009840 comp rx=0 tx=0).stop 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 -- 172.21.15.16:0/2515074534 shutdown_connections 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/2515074534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc2380b3bd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/2515074534 >> v2:172.21.15.177:3300/0 conn(0x7fc2380b14e0 0x7fc2380b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 -- 172.21.15.16:0/2515074534 >> 172.21.15.16:0/2515074534 conn(0x7fc2380686f0 msgr2=0x7fc23806c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 -- 172.21.15.16:0/2515074534 shutdown_connections 2023-10-06T11:43:01.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.639+0000 7fc23e3a6700 1 -- 172.21.15.16:0/2515074534 wait complete. 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.640+0000 7fc23e3a6700 1 Processor -- start 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.640+0000 7fc23e3a6700 1 -- start start 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc238069240 0x7fc23814e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2380b14e0 0x7fc238141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238169d40 con 0x7fc238153cd0 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238156e60 con 0x7fc238069240 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23e3a6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2380c8220 con 0x7fc2380b14e0 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23dba5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23dba5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39502/0 (socket says 172.21.15.16:39502) 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23dba5700 1 -- 172.21.15.16:0/110680946 learned_addr learned my addr 172.21.15.16:0/110680946 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.641+0000 7fc23d3a4700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc238069240 0x7fc23814e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23cba3700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2380b14e0 0x7fc238141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 -- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2380b14e0 msgr2=0x7fc238141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2380b14e0 0x7fc238141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 -- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc238069240 msgr2=0x7fc23814e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc238069240 0x7fc23814e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 -- 172.21.15.16:0/110680946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228009ae0 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23dba5700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c560 tx=0x7fc230007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230011590 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2300116f0 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc238157100 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230011ab0 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.642+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2380c7ef0 con 0x7fc238153cd0 2023-10-06T11:43:01.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.643+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc238157620 con 0x7fc238153cd0 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.643+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fc23001b3b0 con 0x7fc238153cd0 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.644+0000 7fc2357fa700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.644+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc230083d60 con 0x7fc238153cd0 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.644+0000 7fc23d3a4700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.644+0000 7fc23d3a4700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 0x7fc224069590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc228009880 tx=0x7fc228018000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.648+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc238157620 con 0x7fc238153cd0 2023-10-06T11:43:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.845+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc238157620 con 0x7fc238153cd0 2023-10-06T11:43:01.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.845+0000 7fc2357fa700 1 -- 172.21.15.16:0/110680946 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc238157620 con 0x7fc238153cd0 2023-10-06T11:43:01.848 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 msgr2=0x7fc224069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 0x7fc224069590 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc228009880 tx=0x7fc228018000 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 msgr2=0x7fc238142270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc23000c560 tx=0x7fc230007ef0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 shutdown_connections 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc238153cd0 0x7fc238142270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc238069240 0x7fc23814e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 --2- 172.21.15.16:0/110680946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc2380b14e0 0x7fc238141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 >> 172.21.15.16:0/110680946 conn(0x7fc2380686f0 msgr2=0x7fc238152a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 shutdown_connections 2023-10-06T11:43:01.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.847+0000 7fc23e3a6700 1 -- 172.21.15.16:0/110680946 wait complete. 2023-10-06T11:43:01.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T11:43:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.983+0000 7f75ebdbb700 1 Processor -- start 2023-10-06T11:43:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.983+0000 7f75ebdbb700 1 -- start start 2023-10-06T11:43:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ebdbb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e41505b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ebdbb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75e4150af0 0x7f75e4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ebdbb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e40c8bf0 con 0x7f75e40ad100 2023-10-06T11:43:01.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ebdbb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e4157c80 con 0x7f75e4150af0 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ebdbb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e406b380 con 0x7f75e406af30 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75e4150af0 0x7f75e4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75e4150af0 0x7f75e4157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39516/0 (socket says 172.21.15.16:39516) 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 -- 172.21.15.16:0/2390802627 learned_addr learned my addr 172.21.15.16:0/2390802627 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75e9356700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e41505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 -- 172.21.15.16:0/2390802627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 msgr2=0x7f75e41505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e41505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 -- 172.21.15.16:0/2390802627 >> v1:172.21.15.177:6789/0 conn(0x7f75e40ad100 legacy=0x7f75e40ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75ea358700 1 -- 172.21.15.16:0/2390802627 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4069e90 con 0x7f75e4150af0 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.984+0000 7f75e9356700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e41505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ea358700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4150af0 0x7f75e4157e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0099b0 tx=0x7f75dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f056aa73d25f2fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75e8b55700 1 -- 172.21.15.16:0/2390802627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00b860 con 0x7f75e4150af0 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75e8b55700 1 -- 172.21.15.16:0/2390802627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc00b9c0 con 0x7f75e4150af0 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75e8b55700 1 -- 172.21.15.16:0/2390802627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00bd60 con 0x7f75e4150af0 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 -- 172.21.15.16:0/2390802627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4150af0 msgr2=0x7f75e4157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4150af0 0x7f75e4157e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0099b0 tx=0x7f75dc009250 comp rx=0 tx=0).stop 2023-10-06T11:43:01.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 -- 172.21.15.16:0/2390802627 shutdown_connections 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e4150af0 0x7f75e4157e40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/2390802627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e41505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 -- 172.21.15.16:0/2390802627 >> 172.21.15.16:0/2390802627 conn(0x7f75e406ce90 msgr2=0x7f75e40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 -- 172.21.15.16:0/2390802627 shutdown_connections 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.985+0000 7f75ebdbb700 1 -- 172.21.15.16:0/2390802627 wait complete. 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 Processor -- start 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 -- start start 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 0x7f75e415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e414e760 con 0x7f75e40ad100 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e416ab80 con 0x7f75e4150af0 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75ebdbb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e40ac050 con 0x7f75e406af30 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75e9356700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75e9356700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39526/0 (socket says 172.21.15.16:39526) 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.987+0000 7f75e9356700 1 -- 172.21.15.16:0/1703036776 learned_addr learned my addr 172.21.15.16:0/1703036776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75ea358700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 0x7f75e415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9b57700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 -- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 msgr2=0x7f75e414e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e414e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 -- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 msgr2=0x7f75e415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 0x7f75e415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 -- 172.21.15.16:0/1703036776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0094f0 con 0x7f75e40ad100 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75ea358700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 0x7f75e415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e9356700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d800e900 tx=0x7f75d8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800eea0 con 0x7f75e40ad100 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d800c5e0 con 0x7f75e40ad100 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e4169f70 con 0x7f75e40ad100 2023-10-06T11:43:01.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800c980 con 0x7f75e40ad100 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.988+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e416a530 con 0x7f75e40ad100 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.989+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e4151410 con 0x7f75e40ad100 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.990+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f75d800cd30 con 0x7f75e40ad100 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.990+0000 7f75e1ffb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 0x7f75d0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.990+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f75d80848f0 con 0x7f75e40ad100 2023-10-06T11:43:01.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.990+0000 7f75e9b57700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 0x7f75d0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.991+0000 7f75e9b57700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 0x7f75d0069590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75d4007fd0 tx=0x7f75d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:01.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:01.994+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75e4151410 con 0x7f75e40ad100 2023-10-06T11:43:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.189+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75e4151410 con 0x7f75e40ad100 2023-10-06T11:43:02.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.189+0000 7f75e1ffb700 1 -- 172.21.15.16:0/1703036776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v32) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75e4151410 con 0x7f75e40ad100 2023-10-06T11:43:02.193 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-10-06T11:43:02.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 msgr2=0x7f75d0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 0x7f75d0069590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75d4007fd0 tx=0x7f75d4009040 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 msgr2=0x7f75e40a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75d800e900 tx=0x7f75d8008040 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 shutdown_connections 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f75d00670e0 0x7f75d0069590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f75e4150af0 0x7f75e415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f75e406af30 0x7f75e414e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 --2- 172.21.15.16:0/1703036776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75e40ad100 0x7f75e40a5490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 >> 172.21.15.16:0/1703036776 conn(0x7f75e406ce90 msgr2=0x7f75e414ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:02.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 shutdown_connections 2023-10-06T11:43:02.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.192+0000 7f75ebdbb700 1 -- 172.21.15.16:0/1703036776 wait complete. 2023-10-06T11:43:02.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T11:43:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.326+0000 7f57cae81700 1 Processor -- start 2023-10-06T11:43:02.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.326+0000 7f57cae81700 1 -- start start 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57cae81700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c40b09d0 0x7f57c4159b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57cae81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b4790 0x7f57c40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57cae81700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c40b1790 con 0x7f57c4159740 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57cae81700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c40b0e60 con 0x7f57c40b09d0 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57cae81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40b1140 con 0x7f57c40b4790 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c40b09d0 0x7f57c4159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57c40b09d0 0x7f57c4159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39534/0 (socket says 172.21.15.16:39534) 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 -- 172.21.15.16:0/520121999 learned_addr learned my addr 172.21.15.16:0/520121999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c941e700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b4790 0x7f57c40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 -- 172.21.15.16:0/520121999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b4790 msgr2=0x7f57c40b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:02.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b4790 0x7f57c40b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 -- 172.21.15.16:0/520121999 >> v1:172.21.15.177:6789/0 conn(0x7f57c4159740 legacy=0x7f57c40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 -- 172.21.15.16:0/520121999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40b1420 con 0x7f57c40b09d0 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.327+0000 7f57c3fff700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b09d0 0x7f57c4159b10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57b80099b0 tx=0x7f57b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9441c734841cd0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57c37fe700 1 -- 172.21.15.16:0/520121999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8009ea0 con 0x7f57c40b09d0 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57c37fe700 1 -- 172.21.15.16:0/520121999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b800b8f0 con 0x7f57c40b09d0 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57c37fe700 1 -- 172.21.15.16:0/520121999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800bc90 con 0x7f57c40b09d0 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 -- 172.21.15.16:0/520121999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b09d0 msgr2=0x7f57c4159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b09d0 0x7f57c4159b10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f57b80099b0 tx=0x7f57b8009250 comp rx=0 tx=0).stop 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 -- 172.21.15.16:0/520121999 shutdown_connections 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b4790 0x7f57c40b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 --2- 172.21.15.16:0/520121999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b09d0 0x7f57c4159b10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 -- 172.21.15.16:0/520121999 >> 172.21.15.16:0/520121999 conn(0x7f57c4099a40 msgr2=0x7f57c406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 -- 172.21.15.16:0/520121999 shutdown_connections 2023-10-06T11:43:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.328+0000 7f57cae81700 1 -- 172.21.15.16:0/520121999 wait complete. 2023-10-06T11:43:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.329+0000 7f57cae81700 1 Processor -- start 2023-10-06T11:43:02.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.329+0000 7f57cae81700 1 -- start start 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b09d0 0x7f57c416db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 0x7f57c415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c415eb60 con 0x7f57c40b4790 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c4153e40 con 0x7f57c4159740 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57cae81700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c416e0b0 con 0x7f57c40b09d0 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39538/0 (socket says 172.21.15.16:39538) 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 -- 172.21.15.16:0/2317277734 learned_addr learned my addr 172.21.15.16:0/2317277734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c941e700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 0x7f57c415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c8c1d700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b09d0 0x7f57c416db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 -- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b09d0 msgr2=0x7f57c416db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b09d0 0x7f57c416db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.330+0000 7f57c3fff700 1 -- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 msgr2=0x7f57c415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c3fff700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 0x7f57c415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c3fff700 1 -- 172.21.15.16:0/2317277734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80094f0 con 0x7f57c40b4790 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c941e700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 0x7f57c415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c3fff700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f57c406d3f0 tx=0x7f57b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b800bb30 con 0x7f57c40b4790 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8026430 con 0x7f57c40b4790 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c415df50 con 0x7f57c40b4790 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80267f0 con 0x7f57c40b4790 2023-10-06T11:43:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.331+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c415e510 con 0x7f57c40b4790 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.332+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c404c380 con 0x7f57c40b4790 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.332+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f57b8023030 con 0x7f57c40b4790 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.333+0000 7f57c0ff9700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 0x7f57b0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.333+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f57b8086220 con 0x7f57c40b4790 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.333+0000 7f57c8c1d700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 0x7f57b0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.333+0000 7f57c8c1d700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 0x7f57b0069590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57b40037c0 tx=0x7f57b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.337+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57b80531e0 con 0x7f57c40b4790 2023-10-06T11:43:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.530+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57c40d0000 con 0x7f57c40b4790 2023-10-06T11:43:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.530+0000 7f57c0ff9700 1 -- 172.21.15.16:0/2317277734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v32) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57b8056800 con 0x7f57c40b4790 2023-10-06T11:43:02.532 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 msgr2=0x7f57b0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 0x7f57b0069590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f57b40037c0 tx=0x7f57b4009040 comp rx=0 tx=0).stop 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 msgr2=0x7f57c414eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f57c406d3f0 tx=0x7f57b8017000 comp rx=0 tx=0).stop 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 shutdown_connections 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f57b00670e0 0x7f57b0069590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f57c4159740 0x7f57c415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f57c40b09d0 0x7f57c416db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 --2- 172.21.15.16:0/2317277734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57c40b4790 0x7f57c414eb80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 >> 172.21.15.16:0/2317277734 conn(0x7f57c4099a40 msgr2=0x7f57c406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 shutdown_connections 2023-10-06T11:43:02.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.532+0000 7f57cae81700 1 -- 172.21.15.16:0/2317277734 wait complete. 2023-10-06T11:43:02.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:43:02.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.664+0000 7f5bc7be5700 1 Processor -- start 2023-10-06T11:43:02.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.664+0000 7f5bc7be5700 1 -- start start 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc7be5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5bc015cc20 0x7f5bc00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc7be5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bc00cd6a0 0x7f5bc00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc7be5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00cdb10 con 0x7f5bc00cf280 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc7be5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3110 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc7be5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00ccb20 con 0x7f5bc015cc20 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc5180700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bc00cd6a0 0x7f5bc00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc5180700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bc00cd6a0 0x7f5bc00ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39546/0 (socket says 172.21.15.16:39546) 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.665+0000 7f5bc5180700 1 -- 172.21.15.16:0/456566801 learned_addr learned my addr 172.21.15.16:0/456566801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5981700 1 --2- 172.21.15.16:0/456566801 >> v2:172.21.15.177:3300/0 conn(0x7f5bc015cc20 0x7f5bc00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5180700 1 -- 172.21.15.16:0/456566801 >> v2:172.21.15.177:3300/0 conn(0x7f5bc015cc20 msgr2=0x7f5bc00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5180700 1 --2- 172.21.15.16:0/456566801 >> v2:172.21.15.177:3300/0 conn(0x7f5bc015cc20 0x7f5bc00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5180700 1 -- 172.21.15.16:0/456566801 >> v1:172.21.15.177:6789/0 conn(0x7f5bc00cf280 legacy=0x7f5bc015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5180700 1 -- 172.21.15.16:0/456566801 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00cbf10 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc5180700 1 --2- 172.21.15.16:0/456566801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00ced40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0089130 tx=0x7f5bb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2c5324b15c5953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc497f700 1 -- 172.21.15.16:0/456566801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400b820 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc497f700 1 -- 172.21.15.16:0/456566801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb400b980 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc497f700 1 -- 172.21.15.16:0/456566801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400bd40 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc7be5700 1 -- 172.21.15.16:0/456566801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cd6a0 msgr2=0x7f5bc00ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.666+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/456566801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00ced40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0089130 tx=0x7f5bb4009250 comp rx=0 tx=0).stop 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 -- 172.21.15.16:0/456566801 shutdown_connections 2023-10-06T11:43:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/456566801 >> v2:172.21.15.177:3300/0 conn(0x7f5bc015cc20 0x7f5bc00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/456566801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00ced40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 -- 172.21.15.16:0/456566801 >> 172.21.15.16:0/456566801 conn(0x7f5bc0099b40 msgr2=0x7f5bc006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 -- 172.21.15.16:0/456566801 shutdown_connections 2023-10-06T11:43:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.667+0000 7f5bc7be5700 1 -- 172.21.15.16:0/456566801 wait complete. 2023-10-06T11:43:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 Processor -- start 2023-10-06T11:43:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 -- start start 2023-10-06T11:43:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 0x7f5bc00b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3b30 con 0x7f5bc00cf280 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0153220 con 0x7f5bc00cd6a0 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.668+0000 7f5bc7be5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3500 con 0x7f5bc015cc20 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39556/0 (socket says 172.21.15.16:39556) 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 -- 172.21.15.16:0/942918821 learned_addr learned my addr 172.21.15.16:0/942918821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5981700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc6182700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 0x7f5bc00b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 -- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 msgr2=0x7f5bc00b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 0x7f5bc00b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 -- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5bc00cd6a0 msgr2=0x7f5bc00b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 -- 172.21.15.16:0/942918821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb40094f0 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc6182700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 0x7f5bc00b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc5180700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0089390 tx=0x7f5bb4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4006ab0 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb4020070 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00b5cd0 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.669+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb401b450 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.670+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc00b37c0 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.671+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00b3ca0 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.671+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f5bb4022030 con 0x7f5bc00cf280 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.671+0000 7f5bbdffb700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 0x7f5bac0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:02.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.671+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5bb40866a0 con 0x7f5bc00cf280 2023-10-06T11:43:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.672+0000 7f5bc5981700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 0x7f5bac0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:02.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.672+0000 7f5bc5981700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 0x7f5bac0695c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.676+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bb4053530 con 0x7f5bc00cf280 2023-10-06T11:43:02.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.921+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bc00b3c70 con 0x7f5bc00cf280 2023-10-06T11:43:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.922+0000 7f5bbdffb700 1 -- 172.21.15.16:0/942918821 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f5bb40092d0 con 0x7f5bc00cf280 2023-10-06T11:43:02.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:02.924 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 msgr2=0x7f5bac0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 0x7f5bac0695c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0006fd0 tx=0x7f5bb0008040 comp rx=0 tx=0).stop 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 msgr2=0x7f5bc0152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0089390 tx=0x7f5bb4016000 comp rx=0 tx=0).stop 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 shutdown_connections 2023-10-06T11:43:02.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f5bac067110 0x7f5bac0695c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5bc015cc20 0x7f5bc00b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5bc00cd6a0 0x7f5bc00b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 --2- 172.21.15.16:0/942918821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bc00cf280 0x7f5bc0152ce0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 >> 172.21.15.16:0/942918821 conn(0x7f5bc0099b40 msgr2=0x7f5bc0154530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 shutdown_connections 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:02.924+0000 7f5bc7be5700 1 -- 172.21.15.16:0/942918821 wait complete. 2023-10-06T11:43:02.927 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-10-06T11:43:02.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.055+0000 7fd0b69a4700 1 Processor -- start 2023-10-06T11:43:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.055+0000 7fd0b69a4700 1 -- start start 2023-10-06T11:43:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0b69a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd0b00cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0b69a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b00ccb20 con 0x7fd0b015cc20 2023-10-06T11:43:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0b69a4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b00cbf10 con 0x7fd0b00cd870 2023-10-06T11:43:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0b69a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00b3110 con 0x7fd0b00c65a0 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0affff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd0b015cc20 0x7fd0b00cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49620/0 (socket says 172.21.15.16:49620) 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0affff700 1 -- 172.21.15.16:0/969239208 learned_addr learned my addr 172.21.15.16:0/969239208 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.056+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd0b00cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197627080 0 0) 0x7fd0b00ccb20 con 0x7fd0b015cc20 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd098003570 con 0x7fd0b015cc20 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276620174 0 0) 0x7fd0b00cbf10 con 0x7fd0b00cd870 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b00ccb20 con 0x7fd0b00cd870 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3447396680 0 0) 0x7fd098003570 con 0x7fd0b015cc20 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 msgr2=0x7fd0b00cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd0b00cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 >> v1:172.21.15.177:6789/0 conn(0x7fd0b00cd870 legacy=0x7fd0b00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00cc1f0 con 0x7fd0b015cc20 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fd0a0001470 con 0x7fd0b015cc20 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 >> v1:172.21.15.16:6789/0 conn(0x7fd0b015cc20 legacy=0x7fd0b00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098003c20 0x7fd098004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd098004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd098004d50 0x7fd0980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00b3110 con 0x7fd098004d50 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09800bc20 con 0x7fd098003c20 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09800bde0 con 0x7fd0b00c65a0 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd098004d50 0x7fd0980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0affff700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098003c20 0x7fd098004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0af7fe700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd098004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 -- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 msgr2=0x7fd098004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd098004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 -- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098003c20 msgr2=0x7fd098004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd098003c20 0x7fd098004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0b4f41700 1 -- 172.21.15.16:0/969239208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098003570 con 0x7fd098004d50 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.057+0000 7fd0af7fe700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b00c65a0 0x7fd098004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd098004d50 0x7fd0980051c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e210 tx=0x7fd0a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce095a6e580689b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800eac0 con 0x7fd098004d50 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a800f040 con 0x7fd098004d50 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0aeffd700 1 -- 172.21.15.16:0/969239208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a800ee70 con 0x7fd098004d50 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 -- 172.21.15.16:0/969239208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd098004d50 msgr2=0x7fd0980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/969239208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd098004d50 0x7fd0980051c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800e210 tx=0x7fd0a800b040 comp rx=0 tx=0).stop 2023-10-06T11:43:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b4f41700 1 -- 172.21.15.16:0/969239208 reap_dead start 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 -- 172.21.15.16:0/969239208 shutdown_connections 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 -- 172.21.15.16:0/969239208 >> 172.21.15.16:0/969239208 conn(0x7fd0b0099b40 msgr2=0x7fd0b006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 -- 172.21.15.16:0/969239208 shutdown_connections 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.058+0000 7fd0b69a4700 1 -- 172.21.15.16:0/969239208 wait complete. 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.059+0000 7fd0b69a4700 1 Processor -- start 2023-10-06T11:43:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 -- start start 2023-10-06T11:43:03.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b0149110 0x7fd0b00b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b015bcb0 0x7fd0b00b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b015c150 con 0x7fd0b01494e0 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00b4f60 con 0x7fd0b015bcb0 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0b69a4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b00b4350 con 0x7fd0b0149110 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0af7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0af7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39562/0 (socket says 172.21.15.16:39562) 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.060+0000 7fd0af7fe700 1 -- 172.21.15.16:0/1049787957 learned_addr learned my addr 172.21.15.16:0/1049787957 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 -- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b0149110 msgr2=0x7fd0b00b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b0149110 0x7fd0b00b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 -- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b015bcb0 msgr2=0x7fd0b00b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0b4f41700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b015bcb0 0x7fd0b00b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b015bcb0 0x7fd0b00b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 -- 172.21.15.16:0/1049787957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b00b4630 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0af7fe700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e7b0 tx=0x7fd0a4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4001640 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4010040 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b00b4910 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40019b0 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.061+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b00b3b20 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.062+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b004c380 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.062+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd0a4001d20 con 0x7fd0b01494e0 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.063+0000 7fd097fff700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 0x7fd09c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.063+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd0a40844a0 con 0x7fd0b01494e0 2023-10-06T11:43:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.063+0000 7fd0affff700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 0x7fd09c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.063+0000 7fd0affff700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 0x7fd09c069590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007d70 tx=0x7fd0a0001e10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.067+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0a404d5a0 con 0x7fd0b01494e0 2023-10-06T11:43:03.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.261+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b00cc4d0 con 0x7fd0b01494e0 2023-10-06T11:43:03.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.262+0000 7fd097fff700 1 -- 172.21.15.16:0/1049787957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16866 (secure 0 0 0) 0x7fd0a40512d0 con 0x7fd0b01494e0 2023-10-06T11:43:03.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:03.265 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:46.714204+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 msgr2=0x7fd09c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 0x7fd09c069590 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007d70 tx=0x7fd0a0001e10 comp rx=0 tx=0).stop 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 msgr2=0x7fd0b015b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400e7b0 tx=0x7fd0a4009040 comp rx=0 tx=0).stop 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 shutdown_connections 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd09c0670e0 0x7fd09c069590 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd0b015bcb0 0x7fd0b00b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0b0149110 0x7fd0b00b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 --2- 172.21.15.16:0/1049787957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0b01494e0 0x7fd0b015b770 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 >> 172.21.15.16:0/1049787957 conn(0x7fd0b0099b40 msgr2=0x7fd0b00c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 shutdown_connections 2023-10-06T11:43:03.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.264+0000 7fd0b69a4700 1 -- 172.21.15.16:0/1049787957 wait complete. 2023-10-06T11:43:03.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:43:03.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:43:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.396+0000 7f174284d700 1 Processor -- start 2023-10-06T11:43:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.396+0000 7f174284d700 1 -- start start 2023-10-06T11:43:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f173c1593c0 0x7f173c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c159790 0x7f173c15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173c0b53f0 0x7f173c0b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c15a650 con 0x7f173c0b53f0 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c0b61c0 con 0x7f173c1593c0 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174284d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c159d20 con 0x7f173c159790 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174204c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173c0b53f0 0x7f173c0b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174204c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173c0b53f0 0x7f173c0b5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39578/0 (socket says 172.21.15.16:39578) 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174204c700 1 -- 172.21.15.16:0/3862881529 learned_addr learned my addr 172.21.15.16:0/3862881529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174104a700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c159790 0x7f173c15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.397+0000 7f174184b700 1 --2- 172.21.15.16:0/3862881529 >> v2:172.21.15.177:3300/0 conn(0x7f173c1593c0 0x7f173c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 -- 172.21.15.16:0/3862881529 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c159790 msgr2=0x7f173c15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c159790 0x7f173c15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 -- 172.21.15.16:0/3862881529 >> v2:172.21.15.177:3300/0 conn(0x7f173c1593c0 msgr2=0x7f173c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 --2- 172.21.15.16:0/3862881529 >> v2:172.21.15.177:3300/0 conn(0x7f173c1593c0 0x7f173c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 -- 172.21.15.16:0/3862881529 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c15a000 con 0x7f173c0b53f0 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174204c700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0b53f0 0x7f173c0b5850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f173c08a760 tx=0x7f1734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9397561f9ad3668a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f1740849700 1 -- 172.21.15.16:0/3862881529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400b640 con 0x7f173c0b53f0 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f1740849700 1 -- 172.21.15.16:0/3862881529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173400b7a0 con 0x7f173c0b53f0 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f1740849700 1 -- 172.21.15.16:0/3862881529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400bb60 con 0x7f173c0b53f0 2023-10-06T11:43:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174284d700 1 -- 172.21.15.16:0/3862881529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0b53f0 msgr2=0x7f173c0b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.398+0000 7f174284d700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0b53f0 0x7f173c0b5850 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f173c08a760 tx=0x7f1734009250 comp rx=0 tx=0).stop 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 -- 172.21.15.16:0/3862881529 shutdown_connections 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0b53f0 0x7f173c0b5850 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 --2- 172.21.15.16:0/3862881529 >> v2:172.21.15.177:3300/0 conn(0x7f173c1593c0 0x7f173c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 --2- 172.21.15.16:0/3862881529 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c159790 0x7f173c15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 -- 172.21.15.16:0/3862881529 >> 172.21.15.16:0/3862881529 conn(0x7f173c08a020 msgr2=0x7f173c099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 -- 172.21.15.16:0/3862881529 shutdown_connections 2023-10-06T11:43:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.399+0000 7f174284d700 1 -- 172.21.15.16:0/3862881529 wait complete. 2023-10-06T11:43:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 Processor -- start 2023-10-06T11:43:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 -- start start 2023-10-06T11:43:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0b53f0 0x7f173c1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 0x7f173c0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ade20 con 0x7f173c1593c0 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ad4f0 con 0x7f173c159790 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.400+0000 7f174284d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ad7d0 con 0x7f173c0b53f0 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39592/0 (socket says 172.21.15.16:39592) 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 -- 172.21.15.16:0/545511803 learned_addr learned my addr 172.21.15.16:0/545511803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 -- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0b53f0 msgr2=0x7f173c1421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0b53f0 0x7f173c1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 -- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 msgr2=0x7f173c0b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174204c700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 0x7f173c0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 0x7f173c0b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 -- 172.21.15.16:0/545511803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340094f0 con 0x7f173c1593c0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174204c700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 0x7f173c0b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174104a700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f173000ea00 tx=0x7f1730009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000c850 con 0x7f173c1593c0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000c9b0 con 0x7f173c1593c0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c0adab0 con 0x7f173c1593c0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000cd30 con 0x7f173c1593c0 2023-10-06T11:43:03.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.401+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c06e8b0 con 0x7f173c1593c0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.402+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c06f440 con 0x7f173c1593c0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.403+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f1730013680 con 0x7f173c1593c0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.403+0000 7f1739ffb700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 0x7f1728069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.403+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17300850a0 con 0x7f173c1593c0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.403+0000 7f174184b700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 0x7f1728069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.404+0000 7f174184b700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 0x7f1728069590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f173c06f770 tx=0x7f172c00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.407+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1730051f30 con 0x7f173c1593c0 2023-10-06T11:43:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.644+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f173c0adc40 con 0x7f173c1593c0 2023-10-06T11:43:03.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.645+0000 7f1739ffb700 1 -- 172.21.15.16:0/545511803 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4594 (secure 0 0 0) 0x7f1730055550 con 0x7f173c1593c0 2023-10-06T11:43:03.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:03.647 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 msgr2=0x7f1728069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 0x7f1728069590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f173c06f770 tx=0x7f172c00b040 comp rx=0 tx=0).stop 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 msgr2=0x7f173c0afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f173000ea00 tx=0x7f1730009120 comp rx=0 tx=0).stop 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 shutdown_connections 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f17280670e0 0x7f1728069590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c159790 0x7f173c0b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0b53f0 0x7f173c1421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 --2- 172.21.15.16:0/545511803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c1593c0 0x7f173c0afd00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 >> 172.21.15.16:0/545511803 conn(0x7f173c08a020 msgr2=0x7f173c15bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 shutdown_connections 2023-10-06T11:43:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.647+0000 7f174284d700 1 -- 172.21.15.16:0/545511803 wait complete. 2023-10-06T11:43:03.651 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-10-06T11:43:03.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:43:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.778+0000 7efefd123700 1 Processor -- start 2023-10-06T11:43:03.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.778+0000 7efefd123700 1 -- start start 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efefd123700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efef8159980 0x7efef80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efefd123700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80cbf20 con 0x7efef81595b0 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efefd123700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80cc200 con 0x7efef80cd850 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efefd123700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80cc4e0 con 0x7efef8159980 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efef6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efef81595b0 0x7efef80cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49626/0 (socket says 172.21.15.16:49626) 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efef6d9d700 1 -- 172.21.15.16:0/3506465734 learned_addr learned my addr 172.21.15.16:0/3506465734 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efef759e700 1 --2- 172.21.15.16:0/3506465734 >> v2:172.21.15.177:3300/0 conn(0x7efef8159980 0x7efef80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139787692 0 0) 0x7efef80cbf20 con 0x7efef81595b0 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.779+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee4003570 con 0x7efef81595b0 2023-10-06T11:43:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3183784990 0 0) 0x7efee4003570 con 0x7efef81595b0 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 >> v2:172.21.15.177:3300/0 conn(0x7efef8159980 msgr2=0x7efef80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 --2- 172.21.15.16:0/3506465734 >> v2:172.21.15.177:3300/0 conn(0x7efef8159980 0x7efef80cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 >> v1:172.21.15.177:6789/0 conn(0x7efef80cd850 legacy=0x7efef80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80cc7c0 con 0x7efef81595b0 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7efee80017f0 con 0x7efef81595b0 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 >> v1:172.21.15.16:6789/0 conn(0x7efef81595b0 legacy=0x7efef80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4003a60 0x7efee4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee4004810 0x7efee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 0x7efee400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80cc4e0 con 0x7efee40051c0 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee400c130 con 0x7efee4003a60 2023-10-06T11:43:03.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee400c2f0 con 0x7efee4004810 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 0x7efee400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef6d9d700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4003a60 0x7efee4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef659c700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee4004810 0x7efee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 -- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee4004810 msgr2=0x7efee4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee4004810 0x7efee4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 -- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4003a60 msgr2=0x7efee4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4003a60 0x7efee4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 -- 172.21.15.16:0/3506465734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4003570 con 0x7efee40051c0 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef659c700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee4004810 0x7efee4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 -- 172.21.15.16:0/3506465734 reap_dead start 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.780+0000 7efef759e700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 0x7efee400ba60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efef0006f00 tx=0x7efef00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a729d018b9d8b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000e710 con 0x7efee40051c0 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef000e870 con 0x7efee40051c0 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efef5d9b700 1 -- 172.21.15.16:0/3506465734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000ec10 con 0x7efee40051c0 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 -- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 msgr2=0x7efee400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 0x7efee400ba60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efef0006f00 tx=0x7efef00091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 -- 172.21.15.16:0/3506465734 shutdown_connections 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 --2- 172.21.15.16:0/3506465734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee40051c0 0x7efee400ba60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 -- 172.21.15.16:0/3506465734 >> 172.21.15.16:0/3506465734 conn(0x7efef8099b40 msgr2=0x7efef8087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 -- 172.21.15.16:0/3506465734 shutdown_connections 2023-10-06T11:43:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.781+0000 7efefd123700 1 -- 172.21.15.16:0/3506465734 wait complete. 2023-10-06T11:43:03.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.782+0000 7efefd123700 1 Processor -- start 2023-10-06T11:43:03.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.782+0000 7efefd123700 1 -- start start 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee40051c0 0x7efef80c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efef816cf10 0x7efef80c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80c7d60 con 0x7efef80c7fe0 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef8153600 con 0x7efef816cf10 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efefd123700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef8169ab0 con 0x7efee40051c0 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39606/0 (socket says 172.21.15.16:39606) 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef759e700 1 -- 172.21.15.16:0/2841495816 learned_addr learned my addr 172.21.15.16:0/2841495816 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef759e700 1 -- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee40051c0 msgr2=0x7efef80c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef6d9d700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee40051c0 0x7efef80c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef659c700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efef816cf10 0x7efef80c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.783+0000 7efef759e700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee40051c0 0x7efef80c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efef759e700 1 -- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efef816cf10 msgr2=0x7efef80c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efef759e700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efef816cf10 0x7efef80c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efef759e700 1 -- 172.21.15.16:0/2841495816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80c9590 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efef759e700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efef000e4b0 tx=0x7efef0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef000e710 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef0013e70 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef816d440 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef001c890 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.784+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef816da00 con 0x7efef80c7fe0 2023-10-06T11:43:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.785+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef8169c40 con 0x7efef80c7fe0 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.785+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7efef001cc80 con 0x7efef80c7fe0 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.786+0000 7efeeeffd700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 0x7efed8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.786+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efef0086500 con 0x7efef80c7fe0 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.786+0000 7efef6d9d700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 0x7efed8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.786+0000 7efef6d9d700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 0x7efed8069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efee8001580 tx=0x7efee800a040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:03.790+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efef00532f0 con 0x7efef80c7fe0 2023-10-06T11:43:04.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.028+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efef804c380 con 0x7efef80c7fe0 2023-10-06T11:43:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.029+0000 7efeeeffd700 1 -- 172.21.15.16:0/2841495816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4594 (secure 0 0 0) 0x7efef0056910 con 0x7efef80c7fe0 2023-10-06T11:43:04.031 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:04.031 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T11:42:45.120937+0000","modified":"2023-10-06T11:42:48.742791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4977},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4977":{"gid":4977,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/1696671501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1696671501},{"type":"v1","addr":"172.21.15.16:6835","nonce":1696671501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 msgr2=0x7efed8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 0x7efed8069590 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efee8001580 tx=0x7efee800a040 comp rx=0 tx=0).stop 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 msgr2=0x7efef8169f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efef000e4b0 tx=0x7efef0019000 comp rx=0 tx=0).stop 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 shutdown_connections 2023-10-06T11:43:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efef80c7fe0 0x7efef8169f40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efed80670e0 0x7efed8069590 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efee40051c0 0x7efef80c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 --2- 172.21.15.16:0/2841495816 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efef816cf10 0x7efef80c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 >> 172.21.15.16:0/2841495816 conn(0x7efef8099b40 msgr2=0x7efef8155000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 shutdown_connections 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.031+0000 7efefd123700 1 -- 172.21.15.16:0/2841495816 wait complete. 2023-10-06T11:43:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2023-10-06T11:43:04.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:04.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.162+0000 7f9eb2005700 1 Processor -- start 2023-10-06T11:43:04.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.162+0000 7f9eb2005700 1 -- start start 2023-10-06T11:43:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9eac06eca0 0x7f9eac06f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9eac06c6c0 0x7f9eac0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac159d70 0x7f9eac15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac1598e0 con 0x7f9eac06c6c0 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac06cb80 con 0x7f9eac06eca0 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.163+0000 7f9eb2005700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac06ce60 con 0x7f9eac159d70 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9eac06c6c0 0x7f9eac0b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9eac06c6c0 0x7f9eac0b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39610/0 (socket says 172.21.15.16:39610) 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 -- 172.21.15.16:0/2815823349 learned_addr learned my addr 172.21.15.16:0/2815823349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:04.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb1003700 1 --2- 172.21.15.16:0/2815823349 >> v2:172.21.15.177:3300/0 conn(0x7f9eac06eca0 0x7f9eac06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb1804700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac159d70 0x7f9eac15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 -- 172.21.15.16:0/2815823349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac159d70 msgr2=0x7f9eac15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac159d70 0x7f9eac15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 -- 172.21.15.16:0/2815823349 >> v2:172.21.15.177:3300/0 conn(0x7f9eac06eca0 msgr2=0x7f9eac06f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 --2- 172.21.15.16:0/2815823349 >> v2:172.21.15.177:3300/0 conn(0x7f9eac06eca0 0x7f9eac06f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 -- 172.21.15.16:0/2815823349 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac06d140 con 0x7f9eac06c6c0 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eb0802700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac06c6c0 0x7f9eac0b0e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00099b0 tx=0x7f9ea0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9452f75d5868e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eabfff700 1 -- 172.21.15.16:0/2815823349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000b820 con 0x7f9eac06c6c0 2023-10-06T11:43:04.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eabfff700 1 -- 172.21.15.16:0/2815823349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000b980 con 0x7f9eac06c6c0 2023-10-06T11:43:04.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.164+0000 7f9eabfff700 1 -- 172.21.15.16:0/2815823349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000bd20 con 0x7f9eac06c6c0 2023-10-06T11:43:04.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 -- 172.21.15.16:0/2815823349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac06c6c0 msgr2=0x7f9eac0b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac06c6c0 0x7f9eac0b0e00 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00099b0 tx=0x7f9ea0009250 comp rx=0 tx=0).stop 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 -- 172.21.15.16:0/2815823349 shutdown_connections 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac159d70 0x7f9eac15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 --2- 172.21.15.16:0/2815823349 >> v2:172.21.15.177:3300/0 conn(0x7f9eac06eca0 0x7f9eac06f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 --2- 172.21.15.16:0/2815823349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac06c6c0 0x7f9eac0b0e00 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 -- 172.21.15.16:0/2815823349 >> 172.21.15.16:0/2815823349 conn(0x7f9eac0ccab0 msgr2=0x7f9eac0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 -- 172.21.15.16:0/2815823349 shutdown_connections 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.165+0000 7f9eb2005700 1 -- 172.21.15.16:0/2815823349 wait complete. 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.166+0000 7f9eb2005700 1 Processor -- start 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.166+0000 7f9eb2005700 1 -- start start 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 0x7f9eac1427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac06eca0 0x7f9eac0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac142d10 con 0x7f9eac159d70 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0af180 con 0x7f9eac06eca0 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb2005700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0a9fb0 con 0x7f9eac06c6c0 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39618/0 (socket says 172.21.15.16:39618) 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 -- 172.21.15.16:0/1775910603 learned_addr learned my addr 172.21.15.16:0/1775910603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb0802700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac06eca0 0x7f9eac0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1003700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 0x7f9eac1427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 -- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 msgr2=0x7f9eac1427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 0x7f9eac1427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 -- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac06eca0 msgr2=0x7f9eac0ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac06eca0 0x7f9eac0ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1804700 1 -- 172.21.15.16:0/1775910603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00094f0 con 0x7f9eac159d70 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.167+0000 7f9eb1003700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 0x7f9eac1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9eb1804700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c880 tx=0x7f9ea4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4011600 con 0x7f9eac159d70 2023-10-06T11:43:04.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea4011760 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0aa290 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4011b00 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.168+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0af460 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.169+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac0aa7b0 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.169+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f9ea4021350 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.170+0000 7f9ea97fa700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 0x7f9e98069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.170+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9ea4084870 con 0x7f9eac159d70 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.170+0000 7f9eb1003700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 0x7f9e98069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.170+0000 7f9eb1003700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 0x7f9e98069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cc250 tx=0x7f9e9c007710 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.174+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9eac0aa7b0 con 0x7f9eac159d70 2023-10-06T11:43:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.368+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9eac0aa7b0 con 0x7f9eac159d70 2023-10-06T11:43:04.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.369+0000 7f9ea97fa700 1 -- 172.21.15.16:0/1775910603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16866 (secure 0 0 0) 0x7f9eac0aa7b0 con 0x7f9eac159d70 2023-10-06T11:43:04.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:04.371 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:42:46.714204+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T11:42:43.414603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T11:42:44.091338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 msgr2=0x7f9e98069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 0x7f9e98069590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0cc250 tx=0x7f9e9c007710 comp rx=0 tx=0).stop 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 msgr2=0x7f9eac0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c880 tx=0x7f9ea4008040 comp rx=0 tx=0).stop 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 shutdown_connections 2023-10-06T11:43:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9eac159d70 0x7f9eac0a5aa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f9e980670e0 0x7f9e98069590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9eac06c6c0 0x7f9eac1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 --2- 172.21.15.16:0/1775910603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9eac06eca0 0x7f9eac0ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 >> 172.21.15.16:0/1775910603 conn(0x7f9eac0ccab0 msgr2=0x7f9eac15c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 shutdown_connections 2023-10-06T11:43:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.371+0000 7f9eb2005700 1 -- 172.21.15.16:0/1775910603 wait complete. 2023-10-06T11:43:04.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T11:43:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.503+0000 7f7f22d98700 1 Processor -- start 2023-10-06T11:43:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.503+0000 7f7f22d98700 1 -- start start 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7f1c0d0a50 0x7f7f1c0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 0x7f7f1c159210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b5c30 con 0x7f7f1c0b5790 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b46f0 con 0x7f7f1c0d0a50 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22d98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b49d0 con 0x7f7f1c0b4260 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39630/0 (socket says 172.21.15.16:39630) 2023-10-06T11:43:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 -- 172.21.15.16:0/4041607893 learned_addr learned my addr 172.21.15.16:0/4041607893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21d96700 1 --2- 172.21.15.16:0/4041607893 >> v2:172.21.15.177:3300/0 conn(0x7f7f1c0d0a50 0x7f7f1c0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22597700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 0x7f7f1c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 -- 172.21.15.16:0/4041607893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 msgr2=0x7f7f1c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 0x7f7f1c159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 -- 172.21.15.16:0/4041607893 >> v2:172.21.15.177:3300/0 conn(0x7f7f1c0d0a50 msgr2=0x7f7f1c0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 --2- 172.21.15.16:0/4041607893 >> v2:172.21.15.177:3300/0 conn(0x7f7f1c0d0a50 0x7f7f1c0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f21595700 1 -- 172.21.15.16:0/4041607893 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0b4cb0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.504+0000 7f7f22597700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 0x7f7f1c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f21595700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c089130 tx=0x7f7f10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d448c8809bb9d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f20d94700 1 -- 172.21.15.16:0/4041607893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f10009e70 con 0x7f7f1c0b5790 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f20d94700 1 -- 172.21.15.16:0/4041607893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1000b860 con 0x7f7f1c0b5790 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f20d94700 1 -- 172.21.15.16:0/4041607893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000bc00 con 0x7f7f1c0b5790 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 -- 172.21.15.16:0/4041607893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 msgr2=0x7f7f1c0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c089130 tx=0x7f7f10009250 comp rx=0 tx=0).stop 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 -- 172.21.15.16:0/4041607893 shutdown_connections 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0b4260 0x7f7f1c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 --2- 172.21.15.16:0/4041607893 >> v2:172.21.15.177:3300/0 conn(0x7f7f1c0d0a50 0x7f7f1c0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 --2- 172.21.15.16:0/4041607893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c0b3d20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 -- 172.21.15.16:0/4041607893 >> 172.21.15.16:0/4041607893 conn(0x7f7f1c0705b0 msgr2=0x7f7f1c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 -- 172.21.15.16:0/4041607893 shutdown_connections 2023-10-06T11:43:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.505+0000 7f7f22d98700 1 -- 172.21.15.16:0/4041607893 wait complete. 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.506+0000 7f7f22d98700 1 Processor -- start 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 -- start start 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 0x7f7f1c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0d0a50 0x7f7f1c16d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c4ed0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c3f40 con 0x7f7f1c0b4260 2023-10-06T11:43:04.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f22d98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c16e0d0 con 0x7f7f1c0d0a50 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f21595700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f21595700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39634/0 (socket says 172.21.15.16:39634) 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.507+0000 7f7f21595700 1 -- 172.21.15.16:0/282204691 learned_addr learned my addr 172.21.15.16:0/282204691 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21d96700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 0x7f7f1c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f22597700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0d0a50 0x7f7f1c16d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 -- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0d0a50 msgr2=0x7f7f1c16d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0d0a50 0x7f7f1c16d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 -- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 msgr2=0x7f7f1c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 0x7f7f1c0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 -- 172.21.15.16:0/282204691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100094f0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21d96700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 0x7f7f1c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f21595700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009290 tx=0x7f7f10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1000bbe0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1001f070 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f100266b0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c16e3b0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.508+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c0c4470 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.509+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c0c4660 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.510+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7f10026a20 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.510+0000 7f7f19ffb700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 0x7f7f08069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.510+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f10021070 con 0x7f7f1c0b5790 2023-10-06T11:43:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.510+0000 7f7f21d96700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 0x7f7f08069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.511+0000 7f7f21d96700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 0x7f7f08069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0076c0 tx=0x7f7f0c00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.514+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f100535a0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.754+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7f1c04c380 con 0x7f7f1c0b5790 2023-10-06T11:43:04.756 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-10-06T11:43:04.755+0000 7f92ac3a7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T11:43:04.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.758+0000 7f7f19ffb700 1 -- 172.21.15.16:0/282204691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7f10056bc0 con 0x7f7f1c0b5790 2023-10-06T11:43:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 msgr2=0x7f7f08069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 0x7f7f08069590 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0076c0 tx=0x7f7f0c00b040 comp rx=0 tx=0).stop 2023-10-06T11:43:04.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 msgr2=0x7f7f1c15b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009290 tx=0x7f7f10017000 comp rx=0 tx=0).stop 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 shutdown_connections 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7f080670e0 0x7f7f08069590 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7f1c0d0a50 0x7f7f1c16d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f1c0b4260 0x7f7f1c0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 --2- 172.21.15.16:0/282204691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f1c0b5790 0x7f7f1c15b1e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 >> 172.21.15.16:0/282204691 conn(0x7f7f1c0705b0 msgr2=0x7f7f1c15a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 shutdown_connections 2023-10-06T11:43:04.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.761+0000 7f7f22d98700 1 -- 172.21.15.16:0/282204691 wait complete. 2023-10-06T11:43:04.765 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T11:43:04.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T11:43:04.798 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-10-06T11:43:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.896+0000 7f21fe762700 1 Processor -- start 2023-10-06T11:43:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.896+0000 7f21fe762700 1 -- start start 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fe762700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fe762700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80cffc0 con 0x7f21f80ced70 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fe762700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80cf3b0 con 0x7f21f80cd670 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fe762700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80cf690 con 0x7f21f81594a0 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fcf5f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21f80ced70 0x7f21f80ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49630/0 (socket says 172.21.15.16:49630) 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fcf5f700 1 -- 172.21.15.16:0/2520023914 learned_addr learned my addr 172.21.15.16:0/2520023914 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21fd760700 1 --2- 172.21.15.16:0/2520023914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178079991 0 0) 0x7f21f80cffc0 con 0x7f21f80ced70 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.897+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0003570 con 0x7f21f80ced70 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11938040 0 0) 0x7f21f80cf3b0 con 0x7f21f80cd670 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21fd760700 1 -- 172.21.15.16:0/2520023914 >> v1:172.21.15.177:6789/0 conn(0x7f21f80cd670 legacy=0x7f21f81631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21fd760700 1 -- 172.21.15.16:0/2520023914 >> v1:172.21.15.16:6789/0 conn(0x7f21f80ced70 legacy=0x7f21f80ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21fd760700 1 -- 172.21.15.16:0/2520023914 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80cf970 con 0x7f21f81594a0 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21fd760700 1 --2- 172.21.15.16:0/2520023914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f80ce830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009ee0 tx=0x7f21e8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6a7f35b5e4da31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e800e070 con 0x7f21f81594a0 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e800b920 con 0x7f21f81594a0 2023-10-06T11:43:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.898+0000 7f21f7fff700 1 -- 172.21.15.16:0/2520023914 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec001470 con 0x7f21f81594a0 2023-10-06T11:43:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 -- 172.21.15.16:0/2520023914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 msgr2=0x7f21f80ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 --2- 172.21.15.16:0/2520023914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f80ce830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009ee0 tx=0x7f21e8009780 comp rx=0 tx=0).stop 2023-10-06T11:43:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 -- 172.21.15.16:0/2520023914 shutdown_connections 2023-10-06T11:43:04.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 --2- 172.21.15.16:0/2520023914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f80ce830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 -- 172.21.15.16:0/2520023914 >> 172.21.15.16:0/2520023914 conn(0x7f21f8089f90 msgr2=0x7f21f80998b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 -- 172.21.15.16:0/2520023914 shutdown_connections 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.899+0000 7f21fe762700 1 -- 172.21.15.16:0/2520023914 wait complete. 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.900+0000 7f21fe762700 1 Processor -- start 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.900+0000 7f21fe762700 1 -- start start 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 0x7f21f81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f809d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8156d10 con 0x7f21f80cd670 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8155ea0 con 0x7f21f80ced70 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8156100 con 0x7f21f81594a0 2023-10-06T11:43:04.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39672/0 (socket says 172.21.15.16:39672) 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 -- 172.21.15.16:0/4125542612 learned_addr learned my addr 172.21.15.16:0/4125542612 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fcf5f700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 0x7f21f81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fdf61700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f809d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 -- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 msgr2=0x7f21f809d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f809d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 -- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 msgr2=0x7f21f81579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 0x7f21f81579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 -- 172.21.15.16:0/4125542612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e8009a20 con 0x7f21f80cd670 2023-10-06T11:43:04.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fdf61700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fcf5f700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 0x7f21f81579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fd760700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097c0 tx=0x7f21e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e801f410 con 0x7f21f80cd670 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e801f570 con 0x7f21f80cd670 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f81563e0 con 0x7f21f80cd670 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.901+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e801f8f0 con 0x7f21f80cd670 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.902+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f81569a0 con 0x7f21f80cd670 2023-10-06T11:43:04.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.902+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f8089530 con 0x7f21f80cd670 2023-10-06T11:43:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.903+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f21e801fd20 con 0x7f21f80cd670 2023-10-06T11:43:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.903+0000 7f21f57fa700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 0x7f21e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.903+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f21e8086d10 con 0x7f21f80cd670 2023-10-06T11:43:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.903+0000 7f21fcf5f700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 0x7f21e4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:04.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.904+0000 7f21fcf5f700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 0x7f21e4069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f21ec001ed0 tx=0x7f21ec000bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:04.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:04.907+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21e8053a40 con 0x7f21f80cd670 2023-10-06T11:43:05.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.148+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f21f80cf140 con 0x7f21f80cd670 2023-10-06T11:43:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.759+0000 7f21f57fa700 1 -- 172.21.15.16:0/4125542612 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f21e8057060 con 0x7f21f80cd670 2023-10-06T11:43:05.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 msgr2=0x7f21e4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 0x7f21e4069590 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f21ec001ed0 tx=0x7f21ec000bc0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 msgr2=0x7f21f8146ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21e80097c0 tx=0x7f21e8017000 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 shutdown_connections 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f80cd670 0x7f21f8146ec0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f21e40670e0 0x7f21e4069590 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21f81594a0 0x7f21f809d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 --2- 172.21.15.16:0/4125542612 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21f80ced70 0x7f21f81579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 >> 172.21.15.16:0/4125542612 conn(0x7f21f8089f90 msgr2=0x7f21f806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 shutdown_connections 2023-10-06T11:43:05.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.762+0000 7f21fe762700 1 -- 172.21.15.16:0/4125542612 wait complete. 2023-10-06T11:43:05.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T11:43:05.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.983+0000 7f8269a72700 1 Processor -- start 2023-10-06T11:43:05.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.983+0000 7f8269a72700 1 -- start start 2023-10-06T11:43:05.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8269a72700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 0x7f826415a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8269a72700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8264157370 0x7f82640cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8269a72700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640cebc0 con 0x7f82640b5ee0 2023-10-06T11:43:05.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8269a72700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640cee60 con 0x7f8264157370 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8269a72700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82640cf140 con 0x7f8264156fa0 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8264157370 0x7f82640cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8264157370 0x7f82640cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 -- 172.21.15.16:0/3440606941 learned_addr learned my addr 172.21.15.16:0/3440606941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 -- 172.21.15.16:0/3440606941 >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 msgr2=0x7f826415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8268a70700 1 --2- 172.21.15.16:0/3440606941 >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 0x7f826415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 --2- 172.21.15.16:0/3440606941 >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 0x7f826415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 -- 172.21.15.16:0/3440606941 >> v1:172.21.15.177:6789/0 conn(0x7f82640b5ee0 legacy=0x7f82640d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 -- 172.21.15.16:0/3440606941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640cf420 con 0x7f8264157370 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8268a70700 1 --2- 172.21.15.16:0/3440606941 >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 0x7f826415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f8263fff700 1 --2- 172.21.15.16:0/3440606941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640cfc20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8264088c40 tx=0x7f8250009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a447f5504f599c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:05.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.984+0000 7f82637fe700 1 -- 172.21.15.16:0/3440606941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000b820 con 0x7f8264157370 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f82637fe700 1 -- 172.21.15.16:0/3440606941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000b980 con 0x7f8264157370 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f82637fe700 1 -- 172.21.15.16:0/3440606941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000bd20 con 0x7f8264157370 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 -- 172.21.15.16:0/3440606941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 msgr2=0x7f82640cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 --2- 172.21.15.16:0/3440606941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640cfc20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8264088c40 tx=0x7f8250009250 comp rx=0 tx=0).stop 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 -- 172.21.15.16:0/3440606941 shutdown_connections 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 --2- 172.21.15.16:0/3440606941 >> v2:172.21.15.177:3300/0 conn(0x7f8264156fa0 0x7f826415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 --2- 172.21.15.16:0/3440606941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640cfc20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 -- 172.21.15.16:0/3440606941 >> 172.21.15.16:0/3440606941 conn(0x7f826406cea0 msgr2=0x7f826406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 -- 172.21.15.16:0/3440606941 shutdown_connections 2023-10-06T11:43:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.985+0000 7f8269a72700 1 -- 172.21.15.16:0/3440606941 wait complete. 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.986+0000 7f8269a72700 1 Processor -- start 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.986+0000 7f8269a72700 1 -- start start 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82640b5ee0 0x7f8264152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 0x7f826415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826415bca0 con 0x7f8264157370 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826415ae40 con 0x7f8264156fa0 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269a72700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415b090 con 0x7f82640b5ee0 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269271700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269271700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39690/0 (socket says 172.21.15.16:39690) 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269271700 1 -- 172.21.15.16:0/2386042296 learned_addr learned my addr 172.21.15.16:0/2386042296 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8263fff700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 0x7f826415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269271700 1 -- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82640b5ee0 msgr2=0x7f8264152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8268a70700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82640b5ee0 0x7f8264152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.987+0000 7f8269271700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82640b5ee0 0x7f8264152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269271700 1 -- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 msgr2=0x7f826415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269271700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 0x7f826415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269271700 1 -- 172.21.15.16:0/2386042296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500094f0 con 0x7f8264157370 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8263fff700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 0x7f826415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269271700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c930 tx=0x7f825c0026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00ce70 con 0x7f8264157370 2023-10-06T11:43:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00a7e0 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826415b370 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00aba0 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.988+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826415b930 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.989+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826415be50 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.989+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f825c012420 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.990+0000 7f8260ff9700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 0x7f8248069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.990+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f825c084de0 con 0x7f8264157370 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.990+0000 7f8268a70700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 0x7f8248069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.991+0000 7f8268a70700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 0x7f8248069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8258008a30 tx=0x7f8258008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:05.994+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f825c055760 con 0x7f8264157370 2023-10-06T11:43:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.186+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f826404c380 con 0x7f8264157370 2023-10-06T11:43:06.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.918+0000 7f8260ff9700 1 -- 172.21.15.16:0/2386042296 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) v1 ==== 181+0+0 (secure 0 0 0) 0x7f825c0552f0 con 0x7f8264157370 2023-10-06T11:43:06.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 msgr2=0x7f8248069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 0x7f8248069590 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8258008a30 tx=0x7f8258008040 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 msgr2=0x7f82640b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f825c00c930 tx=0x7f825c0026e0 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 shutdown_connections 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8264157370 0x7f82640b1d70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f82480670e0 0x7f8248069590 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f82640b5ee0 0x7f8264152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 --2- 172.21.15.16:0/2386042296 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8264156fa0 0x7f826415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 >> 172.21.15.16:0/2386042296 conn(0x7f826406cea0 msgr2=0x7f82641545d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 shutdown_connections 2023-10-06T11:43:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:06.921+0000 7f8269a72700 1 -- 172.21.15.16:0/2386042296 wait complete. 2023-10-06T11:43:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-10-06T11:43:06.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T11:43:07.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.067+0000 7fad2ce0b700 1 Processor -- start 2023-10-06T11:43:07.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.067+0000 7fad2ce0b700 1 -- start start 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2ce0b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 0x7fad280b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2ce0b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad280ce110 0x7fad2806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2ce0b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad28159ba0 con 0x7fad280675c0 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2ce0b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280cd570 con 0x7fad280ce110 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2ce0b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad280b0520 con 0x7fad280685e0 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad280ce110 0x7fad2806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad280ce110 0x7fad2806c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39698/0 (socket says 172.21.15.16:39698) 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 -- 172.21.15.16:0/1952916238 learned_addr learned my addr 172.21.15.16:0/1952916238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2659c700 1 --2- 172.21.15.16:0/1952916238 >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 0x7fad280b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:07.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 -- 172.21.15.16:0/1952916238 >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 msgr2=0x7fad280b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 --2- 172.21.15.16:0/1952916238 >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 0x7fad280b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 -- 172.21.15.16:0/1952916238 >> v1:172.21.15.177:6789/0 conn(0x7fad280675c0 legacy=0x7fad280cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 -- 172.21.15.16:0/1952916238 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad280b2bf0 con 0x7fad280ce110 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad2659c700 1 --2- 172.21.15.16:0/1952916238 >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 0x7fad280b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.068+0000 7fad26d9d700 1 --2- 172.21.15.16:0/1952916238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280ce110 0x7fad2806c410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fad2806e050 tx=0x7fad20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e31003bad81804 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2559a700 1 -- 172.21.15.16:0/1952916238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2000e070 con 0x7fad280ce110 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2559a700 1 -- 172.21.15.16:0/1952916238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2000ba10 con 0x7fad280ce110 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2559a700 1 -- 172.21.15.16:0/1952916238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200092e0 con 0x7fad280ce110 2023-10-06T11:43:07.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1952916238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280ce110 msgr2=0x7fad2806c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1952916238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280ce110 0x7fad2806c410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fad2806e050 tx=0x7fad20009840 comp rx=0 tx=0).stop 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1952916238 shutdown_connections 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1952916238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280ce110 0x7fad2806c410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1952916238 >> v2:172.21.15.177:3300/0 conn(0x7fad280685e0 0x7fad280b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1952916238 >> 172.21.15.16:0/1952916238 conn(0x7fad2806d520 msgr2=0x7fad2806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1952916238 shutdown_connections 2023-10-06T11:43:07.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.069+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1952916238 wait complete. 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.070+0000 7fad2ce0b700 1 Processor -- start 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.070+0000 7fad2ce0b700 1 -- start start 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280685e0 0x7fad28169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280ce110 0x7fad2816a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815d0a0 con 0x7fad280675c0 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad28158ac0 con 0x7fad280685e0 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2ce0b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2815c490 con 0x7fad280ce110 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:07.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39712/0 (socket says 172.21.15.16:39712) 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 -- 172.21.15.16:0/1932353809 learned_addr learned my addr 172.21.15.16:0/1932353809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 -- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280ce110 msgr2=0x7fad2816a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280ce110 0x7fad2816a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 -- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280685e0 msgr2=0x7fad28169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280685e0 0x7fad28169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 -- 172.21.15.16:0/1932353809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20009ae0 con 0x7fad280675c0 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.071+0000 7fad2659c700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad1800c830 tx=0x7fad18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.072+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011600 con 0x7fad280675c0 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.072+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18011760 con 0x7fad280675c0 2023-10-06T11:43:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.072+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2815c770 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.072+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18011b20 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.072+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2815cd30 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.073+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad28063e70 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.073+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fad1800a3e0 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.073+0000 7fad167fc700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 0x7fad08069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.073+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5706+0+0 (secure 0 0 0) 0x7fad18084de0 con 0x7fad280675c0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.074+0000 7fad25d9b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 0x7fad08069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:07.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.074+0000 7fad25d9b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 0x7fad08069590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad1c007fd0 tx=0x7fad1c007b20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.078+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad18051cc0 con 0x7fad280675c0 2023-10-06T11:43:07.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.275+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad280b2ed0 con 0x7fad280675c0 2023-10-06T11:43:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.922+0000 7fad167fc700 1 -- 172.21.15.16:0/1932353809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) v1 ==== 169+0+0 (secure 0 0 0) 0x7fad180150d0 con 0x7fad280675c0 2023-10-06T11:43:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 msgr2=0x7fad08069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 0x7fad08069590 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad1c007fd0 tx=0x7fad1c007b20 comp rx=0 tx=0).stop 2023-10-06T11:43:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 msgr2=0x7fad2815be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:07.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fad1800c830 tx=0x7fad18008040 comp rx=0 tx=0).stop 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 shutdown_connections 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad280675c0 0x7fad2815be90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fad080670e0 0x7fad08069590 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad1c007fd0 tx=0x7fad1c007b20 comp rx=0 tx=0).stop 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad280ce110 0x7fad2816a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 --2- 172.21.15.16:0/1932353809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad280685e0 0x7fad28169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 >> 172.21.15.16:0/1932353809 conn(0x7fad2806d520 msgr2=0x7fad280aa730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 shutdown_connections 2023-10-06T11:43:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:07.925+0000 7fad2ce0b700 1 -- 172.21.15.16:0/1932353809 wait complete. 2023-10-06T11:43:07.929 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-10-06T11:43:07.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T11:43:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.062+0000 7f595eb4c700 1 Processor -- start 2023-10-06T11:43:08.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.062+0000 7f595eb4c700 1 -- start start 2023-10-06T11:43:08.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595eb4c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5958159b10 0x7f5958164890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595eb4c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580cead0 con 0x7f5958159740 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595eb4c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580b47b0 con 0x7f59580cda10 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595eb4c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580affc0 con 0x7f5958159b10 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595d0e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5958159b10 0x7f5958164890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595c8e8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5958159740 0x7f59580b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49640/0 (socket says 172.21.15.16:49640) 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595c8e8700 1 -- 172.21.15.16:0/716735600 learned_addr learned my addr 172.21.15.16:0/716735600 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595d0e9700 1 -- 172.21.15.16:0/716735600 >> v1:172.21.15.177:6789/0 conn(0x7f59580cda10 legacy=0x7f59580b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595d0e9700 1 -- 172.21.15.16:0/716735600 >> v1:172.21.15.16:6789/0 conn(0x7f5958159740 legacy=0x7f59580b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595d0e9700 1 -- 172.21.15.16:0/716735600 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580cdec0 con 0x7f5958159b10 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f59577fe700 1 -- 172.21.15.16:0/716735600 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864898955 0 0) 0x7f59580cead0 con 0x7f5958159740 2023-10-06T11:43:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.063+0000 7f595d0e9700 1 --2- 172.21.15.16:0/716735600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f5958164890 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e60 tx=0x7f5950009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7244dd5b452f8c68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f59577fe700 1 -- 172.21.15.16:0/716735600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595000e070 con 0x7f5958159b10 2023-10-06T11:43:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f59577fe700 1 -- 172.21.15.16:0/716735600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595000bc30 con 0x7f5958159b10 2023-10-06T11:43:08.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f59577fe700 1 -- 172.21.15.16:0/716735600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950012430 con 0x7f5958159b10 2023-10-06T11:43:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 -- 172.21.15.16:0/716735600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 msgr2=0x7f5958164890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 --2- 172.21.15.16:0/716735600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f5958164890 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e60 tx=0x7f5950009780 comp rx=0 tx=0).stop 2023-10-06T11:43:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 -- 172.21.15.16:0/716735600 shutdown_connections 2023-10-06T11:43:08.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 --2- 172.21.15.16:0/716735600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f5958164890 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 -- 172.21.15.16:0/716735600 >> 172.21.15.16:0/716735600 conn(0x7f5958099a40 msgr2=0x7f595806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 -- 172.21.15.16:0/716735600 shutdown_connections 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.064+0000 7f595eb4c700 1 -- 172.21.15.16:0/716735600 wait complete. 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.065+0000 7f595eb4c700 1 Processor -- start 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.065+0000 7f595eb4c700 1 -- start start 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 0x7f595815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 0x7f5958157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595815b690 con 0x7f5958159b10 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595815b8c0 con 0x7f59580cda10 2023-10-06T11:43:08.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595eb4c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595815bba0 con 0x7f5958159740 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595d0e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595d0e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39722/0 (socket says 172.21.15.16:39722) 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595d0e9700 1 -- 172.21.15.16:0/833601510 learned_addr learned my addr 172.21.15.16:0/833601510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f595c8e8700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 0x7f595815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.066+0000 7f5957fff700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 0x7f5958157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 -- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 msgr2=0x7f5958157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 0x7f5958157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 -- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 msgr2=0x7f595815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 0x7f595815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 -- 172.21.15.16:0/833601510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950009a20 con 0x7f5958159b10 2023-10-06T11:43:08.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595c8e8700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 0x7f595815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f5957fff700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 0x7f5958157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595d0e9700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5950013000 tx=0x7f5950019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595001b630 con 0x7f5958159b10 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595001b790 con 0x7f5958159b10 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595815be80 con 0x7f5958159b10 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595001bb30 con 0x7f5958159b10 2023-10-06T11:43:08.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.067+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595815c440 con 0x7f5958159b10 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.068+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595804c380 con 0x7f5958159b10 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.068+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f595001c400 con 0x7f5958159b10 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.069+0000 7f5954ff9700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 0x7f5944069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.069+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f59500866f0 con 0x7f5958159b10 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.069+0000 7f595c8e8700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 0x7f5944069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.069+0000 7f595c8e8700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 0x7f5944069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f59480018d0 tx=0x7f5948009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.073+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5950053770 con 0x7f5958159b10 2023-10-06T11:43:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.266+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f59580b63a0 con 0x7f5958159b10 2023-10-06T11:43:08.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.270+0000 7f5954ff9700 1 -- 172.21.15.16:0/833601510 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5950009880 con 0x7f5958159b10 2023-10-06T11:43:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 msgr2=0x7f5944069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 0x7f5944069590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f59480018d0 tx=0x7f5948009040 comp rx=0 tx=0).stop 2023-10-06T11:43:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 msgr2=0x7f595809bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5950013000 tx=0x7f5950019000 comp rx=0 tx=0).stop 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 shutdown_connections 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5958159b10 0x7f595809bae0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f59440670e0 0x7f5944069590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f59580cda10 0x7f595815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 --2- 172.21.15.16:0/833601510 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5958159740 0x7f5958157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 >> 172.21.15.16:0/833601510 conn(0x7f5958099a40 msgr2=0x7f59580b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 shutdown_connections 2023-10-06T11:43:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.272+0000 7f595eb4c700 1 -- 172.21.15.16:0/833601510 wait complete. 2023-10-06T11:43:08.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread' 2023-10-06T11:43:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.403+0000 7fb37d145700 1 Processor -- start 2023-10-06T11:43:08.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.403+0000 7fb37d145700 1 -- start start 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb37d145700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb378159980 0x7fb3780cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb37d145700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3780cbf20 con 0x7fb3781595b0 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb37d145700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3780cc200 con 0x7fb3780cd850 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb37d145700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3780cc4e0 con 0x7fb378159980 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb376d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb3781595b0 0x7fb3780cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42816/0 (socket says 172.21.15.16:42816) 2023-10-06T11:43:08.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb376d9d700 1 -- 172.21.15.16:0/3439716958 learned_addr learned my addr 172.21.15.16:0/3439716958 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb37759e700 1 --2- 172.21.15.16:0/3439716958 >> v2:172.21.15.177:3300/0 conn(0x7fb378159980 0x7fb3780cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230595599 0 0) 0x7fb3780cbf20 con 0x7fb3781595b0 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb364003570 con 0x7fb3781595b0 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089638660 0 0) 0x7fb364003570 con 0x7fb3781595b0 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 >> v2:172.21.15.177:3300/0 conn(0x7fb378159980 msgr2=0x7fb3780cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 --2- 172.21.15.16:0/3439716958 >> v2:172.21.15.177:3300/0 conn(0x7fb378159980 0x7fb3780cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 >> v1:172.21.15.177:6789/0 conn(0x7fb3780cd850 legacy=0x7fb3780cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780cc7c0 con 0x7fb3781595b0 2023-10-06T11:43:08.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fb368001470 con 0x7fb3781595b0 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 >> v1:172.21.15.16:6789/0 conn(0x7fb3781595b0 legacy=0x7fb3780cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb364003a60 0x7fb364003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb364004810 0x7fb364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 0x7fb36400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780cc4e0 con 0x7fb3640051c0 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36400c160 con 0x7fb364004810 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.404+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36400c320 con 0x7fb364003a60 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 0x7fb36400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37659c700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb364004810 0x7fb364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb376d9d700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb364003a60 0x7fb364003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 -- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb364003a60 msgr2=0x7fb364003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:08.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb364003a60 0x7fb364003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 -- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb364004810 msgr2=0x7fb364004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb364004810 0x7fb364004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 -- 172.21.15.16:0/3439716958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364003570 con 0x7fb3640051c0 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 -- 172.21.15.16:0/3439716958 reap_dead start 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb37759e700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 0x7fb36400ba60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e210 tx=0x7fb37000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbcb2f13e1e61f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000eac0 con 0x7fb3640051c0 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000f040 con 0x7fb3640051c0 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.405+0000 7fb375d9b700 1 -- 172.21.15.16:0/3439716958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000ee70 con 0x7fb3640051c0 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 -- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 msgr2=0x7fb36400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 0x7fb36400ba60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e210 tx=0x7fb37000b040 comp rx=0 tx=0).stop 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 -- 172.21.15.16:0/3439716958 shutdown_connections 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 --2- 172.21.15.16:0/3439716958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3640051c0 0x7fb36400ba60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 -- 172.21.15.16:0/3439716958 >> 172.21.15.16:0/3439716958 conn(0x7fb378099b40 msgr2=0x7fb378087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 -- 172.21.15.16:0/3439716958 shutdown_connections 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.406+0000 7fb37d145700 1 -- 172.21.15.16:0/3439716958 wait complete. 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.407+0000 7fb37d145700 1 Processor -- start 2023-10-06T11:43:08.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.407+0000 7fb37d145700 1 -- start start 2023-10-06T11:43:08.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3640051c0 0x7fb3780ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb37809bf40 0x7fb3781471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780caab0 con 0x7fb37809baf0 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3780ca7e0 con 0x7fb37809bf40 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3781476f0 con 0x7fb3640051c0 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43570/0 (socket says 172.21.15.16:43570) 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 -- 172.21.15.16:0/43104980 learned_addr learned my addr 172.21.15.16:0/43104980 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb376d9d700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3640051c0 0x7fb3780ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37759e700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb37809bf40 0x7fb3781471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 -- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3640051c0 msgr2=0x7fb3780ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3640051c0 0x7fb3780ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 -- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb37809bf40 msgr2=0x7fb3781471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb37809bf40 0x7fb3781471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 -- 172.21.15.16:0/43104980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780b3ae0 con 0x7fb37809baf0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37659c700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb360002950 tx=0x7fb360007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600032e0 con 0x7fb37809baf0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360006820 con 0x7fb37809baf0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.408+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3780b3dc0 con 0x7fb37809baf0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.409+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360006bc0 con 0x7fb37809baf0 2023-10-06T11:43:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.409+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3780b4380 con 0x7fb37809baf0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.409+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3780b45a0 con 0x7fb37809baf0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.410+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb360007030 con 0x7fb37809baf0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.410+0000 7fb36effd700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.410+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb360084640 con 0x7fb37809baf0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.411+0000 7fb376d9d700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.411+0000 7fb376d9d700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 0x7fb358069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb368001050 tx=0x7fb368001e50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.415+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3600517d0 con 0x7fb37809baf0 2023-10-06T11:43:08.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.695+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]} v 0) v1 -- 0x7fb37804c380 con 0x7fb37809baf0 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.924+0000 7fb36effd700 1 -- 172.21.15.16:0/43104980 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestPerModuleFinsherThread.test_volumes_module_finisher_thread"]}]=0 v88) v1 ==== 167+0+0 (secure 0 0 0) 0x7fb360054df0 con 0x7fb37809baf0 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.926+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 msgr2=0x7fb358069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.926+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 0x7fb358069590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb368001050 tx=0x7fb368001e50 comp rx=0 tx=0).stop 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.926+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 msgr2=0x7fb378146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.926+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb360002950 tx=0x7fb360007d30 comp rx=0 tx=0).stop 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 shutdown_connections 2023-10-06T11:43:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb3580670e0 0x7fb358069590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb37809bf40 0x7fb3781471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb3640051c0 0x7fb3780ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 --2- 172.21.15.16:0/43104980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37809baf0 0x7fb378146c70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 >> 172.21.15.16:0/43104980 conn(0x7fb378099b40 msgr2=0x7fb378154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 shutdown_connections 2023-10-06T11:43:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:08.927+0000 7fb37d145700 1 -- 172.21.15.16:0/43104980 wait complete. 2023-10-06T11:43:09.342 INFO:tasks.cephfs_test_runner:test_volumes_module_finisher_thread (tasks.cephfs.test_volumes.TestPerModuleFinsherThread) ... ok 2023-10-06T11:43:09.342 INFO:tasks.cephfs_test_runner: 2023-10-06T11:43:09.343 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T11:43:09.343 INFO:tasks.cephfs_test_runner:Ran 1 test in 41.224s 2023-10-06T11:43:09.343 INFO:tasks.cephfs_test_runner: 2023-10-06T11:43:09.343 INFO:tasks.cephfs_test_runner:OK 2023-10-06T11:43:09.343 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T11:43:09.352 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-06T11:43:09.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:09.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.491+0000 7fdb91f66700 1 Processor -- start 2023-10-06T11:43:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.491+0000 7fdb91f66700 1 -- start start 2023-10-06T11:43:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb91f66700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb91f66700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb91f66700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c0b5c00 con 0x7fdb8c0b4260 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb91f66700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0b46f0 con 0x7fdb8c0b5790 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb91f66700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0b49d0 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb8bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.492+0000 7fdb90f64700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb91765700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fdb8c0b4260 0x7fdb8c159210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:51420/0 (socket says 172.21.15.16:51420) 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb90f64700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:43810/0 (socket says 172.21.15.16:43810) 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45666/0 (socket says 172.21.15.16:45666) 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb91765700 1 -- 172.21.15.16:0/1307019804 learned_addr learned my addr 172.21.15.16:0/1307019804 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:09.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 -- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 msgr2=0x7fdb8c0d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 -- 172.21.15.16:0/1307019804 >> v1:172.21.15.177:6789/0 conn(0x7fdb8c0b4260 legacy=0x7fdb8c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 -- 172.21.15.16:0/1307019804 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0b4cb0 con 0x7fdb8c0b5790 2023-10-06T11:43:09.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb90f64700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8bfff700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb800098e0 tx=0x7fdb800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98f90cb3297feed8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8b7fe700 1 -- 172.21.15.16:0/1307019804 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000b640 con 0x7fdb8c0b5790 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8b7fe700 1 -- 172.21.15.16:0/1307019804 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8000b7a0 con 0x7fdb8c0b5790 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.493+0000 7fdb8b7fe700 1 -- 172.21.15.16:0/1307019804 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000bb40 con 0x7fdb8c0b5790 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 -- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 msgr2=0x7fdb8c0b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdb800098e0 tx=0x7fdb800091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 -- 172.21.15.16:0/1307019804 shutdown_connections 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0d0a50 0x7fdb8c0d0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 --2- 172.21.15.16:0/1307019804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0b3d20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 -- 172.21.15.16:0/1307019804 >> 172.21.15.16:0/1307019804 conn(0x7fdb8c0705b0 msgr2=0x7fdb8c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 -- 172.21.15.16:0/1307019804 shutdown_connections 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.494+0000 7fdb91f66700 1 -- 172.21.15.16:0/1307019804 wait complete. 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.495+0000 7fdb91f66700 1 Processor -- start 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.495+0000 7fdb91f66700 1 -- start start 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0b4260 0x7fdb8c16a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0ac2b0 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c16af50 con 0x7fdb8c0b5790 2023-10-06T11:43:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91f66700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0aba10 con 0x7fdb8c0b4260 2023-10-06T11:43:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91765700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91765700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43584/0 (socket says 172.21.15.16:43584) 2023-10-06T11:43:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb91765700 1 -- 172.21.15.16:0/3399632499 learned_addr learned my addr 172.21.15.16:0/3399632499 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb90f64700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0b4260 0x7fdb8c16a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.496+0000 7fdb8bfff700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 -- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0b4260 msgr2=0x7fdb8c16a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0b4260 0x7fdb8c16a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 -- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 msgr2=0x7fdb8c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 -- 172.21.15.16:0/3399632499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009420 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb8bfff700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91765700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400cbc0 tx=0x7fdb84001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400a900 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8400aa60 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb80009420 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400ade0 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.497+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c0abfd0 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.498+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c16ad20 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.498+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fdb84007300 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.499+0000 7fdb88ff9700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 0x7fdb780695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.499+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fdb84084a70 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.499+0000 7fdb90f64700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 0x7fdb780695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.499+0000 7fdb90f64700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 0x7fdb780695c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0093f0 tx=0x7fdb7c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.503+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb8c16ad20 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.698+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb8c16ad20 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.699+0000 7fdb88ff9700 1 -- 172.21.15.16:0/3399632499 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13779 (secure 0 0 0) 0x7fdb8c16ad20 con 0x7fdb8c0d0a50 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:09.789 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:43:07.918376+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1696671501":"2023-10-07T11:43:04.756021+0000","172.21.15.16:6833/1696671501":"2023-10-07T11:43:04.756021+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 msgr2=0x7fdb780695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 0x7fdb780695c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0093f0 tx=0x7fdb7c008040 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 msgr2=0x7fdb8c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400cbc0 tx=0x7fdb84001990 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 shutdown_connections 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb8c0d0a50 0x7fdb8c0cf800 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdb78067110 0x7fdb780695c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdb8c0b4260 0x7fdb8c16a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 --2- 172.21.15.16:0/3399632499 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb8c0b5790 0x7fdb8c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 >> 172.21.15.16:0/3399632499 conn(0x7fdb8c0705b0 msgr2=0x7fdb8c0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 shutdown_connections 2023-10-06T11:43:09.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.701+0000 7fdb91f66700 1 -- 172.21.15.16:0/3399632499 wait complete. 2023-10-06T11:43:09.792 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T11:41:48.881552+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T11:41:50.113038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T11:43:09.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T11:43:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.913+0000 7fdf352d0700 1 Processor -- start 2023-10-06T11:43:09.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.913+0000 7fdf352d0700 1 -- start start 2023-10-06T11:43:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf352d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf3015bba0 0x7fdf300cb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf352d0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf300cd070 0x7fdf3015a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf352d0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf300ccbe0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf352d0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300ca160 con 0x7fdf3015bba0 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf352d0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf300cc2d0 con 0x7fdf300cd070 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf2e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf3015bba0 0x7fdf300cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf2e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf3015bba0 0x7fdf300cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43592/0 (socket says 172.21.15.16:43592) 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.914+0000 7fdf2e7fc700 1 -- 172.21.15.16:0/2183107045 learned_addr learned my addr 172.21.15.16:0/2183107045 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2f7fe700 1 --2- 172.21.15.16:0/2183107045 >> v2:172.21.15.177:3300/0 conn(0x7fdf300cd070 0x7fdf3015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2e7fc700 1 -- 172.21.15.16:0/2183107045 >> v2:172.21.15.177:3300/0 conn(0x7fdf300cd070 msgr2=0x7fdf3015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2e7fc700 1 --2- 172.21.15.16:0/2183107045 >> v2:172.21.15.177:3300/0 conn(0x7fdf300cd070 0x7fdf3015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2e7fc700 1 -- 172.21.15.16:0/2183107045 >> v1:172.21.15.177:6789/0 conn(0x7fdf300cb0b0 legacy=0x7fdf300c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2e7fc700 1 -- 172.21.15.16:0/2183107045 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300c9310 con 0x7fdf3015bba0 2023-10-06T11:43:09.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2e7fc700 1 --2- 172.21.15.16:0/2183107045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3015bba0 0x7fdf300cb480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b44b0 tx=0x7fdf18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7806e4f03ca0db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2dffb700 1 -- 172.21.15.16:0/2183107045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800b860 con 0x7fdf3015bba0 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2dffb700 1 -- 172.21.15.16:0/2183107045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800b9c0 con 0x7fdf3015bba0 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.915+0000 7fdf2dffb700 1 -- 172.21.15.16:0/2183107045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800bd60 con 0x7fdf3015bba0 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 -- 172.21.15.16:0/2183107045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3015bba0 msgr2=0x7fdf300cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 --2- 172.21.15.16:0/2183107045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3015bba0 0x7fdf300cb480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b44b0 tx=0x7fdf18009250 comp rx=0 tx=0).stop 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 -- 172.21.15.16:0/2183107045 shutdown_connections 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 --2- 172.21.15.16:0/2183107045 >> v2:172.21.15.177:3300/0 conn(0x7fdf300cd070 0x7fdf3015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 --2- 172.21.15.16:0/2183107045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf3015bba0 0x7fdf300cb480 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 -- 172.21.15.16:0/2183107045 >> 172.21.15.16:0/2183107045 conn(0x7fdf30067c90 msgr2=0x7fdf30068090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 -- 172.21.15.16:0/2183107045 shutdown_connections 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.916+0000 7fdf352d0700 1 -- 172.21.15.16:0/2183107045 wait complete. 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.917+0000 7fdf352d0700 1 Processor -- start 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.917+0000 7fdf352d0700 1 -- start start 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf300cd070 0x7fdf30157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 0x7fdf300c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30159ae0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf30167ff0 con 0x7fdf300cd070 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf301682d0 con 0x7fdf3015bba0 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43604/0 (socket says 172.21.15.16:43604) 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 -- 172.21.15.16:0/1791505269 learned_addr learned my addr 172.21.15.16:0/1791505269 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2e7fc700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf300cd070 0x7fdf30157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2f7fe700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 0x7fdf300c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 -- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 msgr2=0x7fdf300c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 0x7fdf300c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 -- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf300cd070 msgr2=0x7fdf30157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:09.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf300cd070 0x7fdf30157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 -- 172.21.15.16:0/1791505269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180094f0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2f7fe700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 0x7fdf300c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf2effd700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c590 tx=0x7fdf2800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28011590 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28011710 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf301685b0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.918+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28011ad0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.919+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf300a59d0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.919+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30168a50 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.920+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fdf2801b3b0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.920+0000 7fdf26ffd700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 0x7fdf10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:09.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.920+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fdf28083be0 con 0x7fdf300cb0b0 2023-10-06T11:43:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.920+0000 7fdf2e7fc700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 0x7fdf10069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.921+0000 7fdf2e7fc700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 0x7fdf10069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b44b0 tx=0x7fdf18017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:09.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:09.925+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf28050d70 con 0x7fdf300cb0b0 2023-10-06T11:43:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.116+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf3004c380 con 0x7fdf300cb0b0 2023-10-06T11:43:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.116+0000 7fdf26ffd700 1 -- 172.21.15.16:0/1791505269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf28017070 con 0x7fdf300cb0b0 2023-10-06T11:43:10.119 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.118+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 msgr2=0x7fdf10069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.118+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 0x7fdf10069590 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdf300b44b0 tx=0x7fdf18017000 comp rx=0 tx=0).stop 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.118+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 msgr2=0x7fdf300a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.118+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c590 tx=0x7fdf2800c8a0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 shutdown_connections 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf300cb0b0 0x7fdf300a5490 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fdf100670e0 0x7fdf10069590 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf3015bba0 0x7fdf300c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 --2- 172.21.15.16:0/1791505269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf300cd070 0x7fdf30157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 >> 172.21.15.16:0/1791505269 conn(0x7fdf30067c90 msgr2=0x7fdf30155490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 shutdown_connections 2023-10-06T11:43:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.119+0000 7fdf352d0700 1 -- 172.21.15.16:0/1791505269 wait complete. 2023-10-06T11:43:10.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T11:43:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.253+0000 7f969ccb4700 1 Processor -- start 2023-10-06T11:43:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.253+0000 7f969ccb4700 1 -- start start 2023-10-06T11:43:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969ccb4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9698159740 0x7f96980b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969ccb4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96980b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969ccb4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980cead0 con 0x7f9698159b10 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969ccb4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980affc0 con 0x7f9698159740 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969ccb4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cdec0 con 0x7f96980cda10 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9698159740 0x7f96980b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9698159740 0x7f96980b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43620/0 (socket says 172.21.15.16:43620) 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f969659c700 1 -- 172.21.15.16:0/3677946995 learned_addr learned my addr 172.21.15.16:0/3677946995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.254+0000 7f9695d9b700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96980b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969659c700 1 -- 172.21.15.16:0/3677946995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 msgr2=0x7f96980b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969659c700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96980b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969659c700 1 -- 172.21.15.16:0/3677946995 >> v1:172.21.15.177:6789/0 conn(0x7f9698159b10 legacy=0x7f969815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969659c700 1 -- 172.21.15.16:0/3677946995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980ce1a0 con 0x7f9698159740 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969659c700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96900099b0 tx=0x7f9690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800fc39615a69a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969559a700 1 -- 172.21.15.16:0/3677946995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000b860 con 0x7f9698159740 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969559a700 1 -- 172.21.15.16:0/3677946995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000b9c0 con 0x7f9698159740 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.255+0000 7f969559a700 1 -- 172.21.15.16:0/3677946995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000bd60 con 0x7f9698159740 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 -- 172.21.15.16:0/3677946995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 msgr2=0x7f96980b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f96900099b0 tx=0x7f9690009250 comp rx=0 tx=0).stop 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 -- 172.21.15.16:0/3677946995 shutdown_connections 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 --2- 172.21.15.16:0/3677946995 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96980b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 -- 172.21.15.16:0/3677946995 >> 172.21.15.16:0/3677946995 conn(0x7f9698099a40 msgr2=0x7f969806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 -- 172.21.15.16:0/3677946995 shutdown_connections 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.256+0000 7f969ccb4700 1 -- 172.21.15.16:0/3677946995 wait complete. 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 Processor -- start 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 -- start start 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96981552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9698159b10 0x7f969815f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980ae9b0 con 0x7f9698159740 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980af210 con 0x7f9698159b10 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.257+0000 7f969ccb4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980b49c0 con 0x7f96980cda10 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43626/0 (socket says 172.21.15.16:43626) 2023-10-06T11:43:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 -- 172.21.15.16:0/2326893256 learned_addr learned my addr 172.21.15.16:0/2326893256 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9696d9d700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9698159b10 0x7f969815f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f969659c700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96981552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 -- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 msgr2=0x7f96981552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96981552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 -- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9698159b10 msgr2=0x7f969815f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9698159b10 0x7f969815f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 -- 172.21.15.16:0/2326893256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900094f0 con 0x7f9698159740 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f9695d9b700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f968000d890 tx=0x7f9680008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968000b600 con 0x7f9698159740 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968000b760 con 0x7f9698159740 2023-10-06T11:43:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96980aebc0 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968000bae0 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.258+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96980ac1c0 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.259+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980af3a0 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.260+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f968001e460 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.260+0000 7f968e7fc700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 0x7f9678069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.260+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f9680084d50 con 0x7f9698159740 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.260+0000 7f969659c700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 0x7f9678069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.261+0000 7f969659c700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 0x7f9678069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96900099b0 tx=0x7f9690017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.264+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9680088030 con 0x7f9698159740 2023-10-06T11:43:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.457+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9698159210 con 0x7f9698159740 2023-10-06T11:43:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.458+0000 7f968e7fc700 1 -- 172.21.15.16:0/2326893256 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9680088020 con 0x7f9698159740 2023-10-06T11:43:10.460 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-10-06T11:43:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 msgr2=0x7f9678069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 0x7f9678069590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96900099b0 tx=0x7f9690017000 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 msgr2=0x7f96980b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f968000d890 tx=0x7f9680008040 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 shutdown_connections 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f96780670e0 0x7f9678069590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9698159b10 0x7f969815f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f96980cda10 0x7f96981552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 --2- 172.21.15.16:0/2326893256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9698159740 0x7f96980b3f90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 >> 172.21.15.16:0/2326893256 conn(0x7f9698099a40 msgr2=0x7f96980b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 shutdown_connections 2023-10-06T11:43:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.460+0000 7f969ccb4700 1 -- 172.21.15.16:0/2326893256 wait complete. 2023-10-06T11:43:10.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-06T11:43:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.591+0000 7fb8d0640700 1 Processor -- start 2023-10-06T11:43:10.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.591+0000 7fb8d0640700 1 -- start start 2023-10-06T11:43:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8d0640700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb8c8159980 0x7fb8c80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8d0640700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80cbf20 con 0x7fb8c80cd850 2023-10-06T11:43:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8d0640700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80cc200 con 0x7fb8c81595b0 2023-10-06T11:43:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8d0640700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80cc4e0 con 0x7fb8c8159980 2023-10-06T11:43:10.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8cdbdb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8c80cd850 0x7fb8c80cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42824/0 (socket says 172.21.15.16:42824) 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8cdbdb700 1 -- 172.21.15.16:0/2563460067 learned_addr learned my addr 172.21.15.16:0/2563460067 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/2563460067 >> v2:172.21.15.177:3300/0 conn(0x7fb8c8159980 0x7fb8c80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876519598 0 0) 0x7fb8c80cbf20 con 0x7fb8c80cd850 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.592+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b0003570 con 0x7fb8c80cd850 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946177949 0 0) 0x7fb8b0003570 con 0x7fb8c80cd850 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 >> v2:172.21.15.177:3300/0 conn(0x7fb8c8159980 msgr2=0x7fb8c80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 --2- 172.21.15.16:0/2563460067 >> v2:172.21.15.177:3300/0 conn(0x7fb8c8159980 0x7fb8c80cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 >> v1:172.21.15.177:6789/0 conn(0x7fb8c81595b0 legacy=0x7fb8c80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80cc7c0 con 0x7fb8c80cd850 2023-10-06T11:43:10.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fb8bc001470 con 0x7fb8c80cd850 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/2563460067 >> v2:172.21.15.177:3300/0 conn(0x7fb8c8159980 0x7fb8c80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 >> v1:172.21.15.16:6789/0 conn(0x7fb8c80cd850 legacy=0x7fb8c80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b0003a60 0x7fb8b0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 0x7fb8b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b00051c0 0x7fb8b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80cc4e0 con 0x7fb8b0004810 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000c160 con 0x7fb8b0003a60 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b000c320 con 0x7fb8b00051c0 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 0x7fb8b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8ce3dc700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b0003a60 0x7fb8b0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b00051c0 0x7fb8b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 -- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b00051c0 msgr2=0x7fb8b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b00051c0 0x7fb8b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 -- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b0003a60 msgr2=0x7fb8b0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b0003a60 0x7fb8b0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 -- 172.21.15.16:0/2563460067 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0003570 con 0x7fb8b0004810 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8ce3dc700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8b0003a60 0x7fb8b0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cebdd700 1 -- 172.21.15.16:0/2563460067 reap_dead start 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cdbdb700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 0x7fb8b0004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c390 tx=0x7fb8bc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f569fb202c3f9e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00cc60 con 0x7fb8b0004810 2023-10-06T11:43:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00cdc0 con 0x7fb8b0004810 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.593+0000 7fb8cd3da700 1 -- 172.21.15.16:0/2563460067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc005b50 con 0x7fb8b0004810 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 -- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 msgr2=0x7fb8b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 0x7fb8b0004c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c390 tx=0x7fb8bc001e50 comp rx=0 tx=0).stop 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 -- 172.21.15.16:0/2563460067 shutdown_connections 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 --2- 172.21.15.16:0/2563460067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b0004810 0x7fb8b0004c80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 -- 172.21.15.16:0/2563460067 >> 172.21.15.16:0/2563460067 conn(0x7fb8c8099b40 msgr2=0x7fb8c8087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 -- 172.21.15.16:0/2563460067 shutdown_connections 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.594+0000 7fb8d0640700 1 -- 172.21.15.16:0/2563460067 wait complete. 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.595+0000 7fb8d0640700 1 Processor -- start 2023-10-06T11:43:10.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.595+0000 7fb8d0640700 1 -- start start 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b0004810 0x7fb8c816e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8c8154300 0x7fb8c8154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80c6c90 con 0x7fb8c80c62d0 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8154cb0 con 0x7fb8c8154300 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8d0640700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8171e50 con 0x7fb8b0004810 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43644/0 (socket says 172.21.15.16:43644) 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 -- 172.21.15.16:0/3276107924 learned_addr learned my addr 172.21.15.16:0/3276107924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 -- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b0004810 msgr2=0x7fb8c816e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b0004810 0x7fb8c816e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 -- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8c8154300 msgr2=0x7fb8c8154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8c8154300 0x7fb8c8154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 -- 172.21.15.16:0/3276107924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8154ee0 con 0x7fb8c80c62d0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.596+0000 7fb8cebdd700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80cf8c0 tx=0x7fb8c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.597+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000e070 con 0x7fb8c80c62d0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.597+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0011db0 con 0x7fb8c80c62d0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.597+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c8171520 con 0x7fb8c80c62d0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.597+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00166e0 con 0x7fb8c80c62d0 2023-10-06T11:43:10.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.597+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c8171ae0 con 0x7fb8c80c62d0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.598+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c804c380 con 0x7fb8c80c62d0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.598+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fb8c0016a50 con 0x7fb8c80c62d0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.598+0000 7fb8c67fc700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 0x7fb8b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.599+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fb8c0085490 con 0x7fb8c80c62d0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.599+0000 7fb8ce3dc700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 0x7fb8b4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.599+0000 7fb8ce3dc700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 0x7fb8b4069590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007fd0 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:10.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.603+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8c0052510 con 0x7fb8c80c62d0 2023-10-06T11:43:10.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.854+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fb8c8171fc0 con 0x7fb8c80c62d0 2023-10-06T11:43:11.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.859+0000 7fb8c67fc700 1 -- 172.21.15.16:0/3276107924 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v7) v1 ==== 81+0+449961 (secure 0 0 0) 0x7fb8c0055b30 con 0x7fb8c80c62d0 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 7, 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 4112, 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2023-10-06T11:43:11.501 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:6832", 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 90263 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:6833", 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 90263 2023-10-06T11:43:11.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.177:6833/90263", 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2023-10-06T11:41:44.052151+0000", 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540138320759226367, 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 4115, 2023-10-06T11:43:11.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540138320759226367, 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.507 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.511 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:11.512 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:11.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.516 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2023-10-06T11:43:11.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2023-10-06T11:43:11.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2023-10-06T11:43:11.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.520 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2023-10-06T11:43:11.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2023-10-06T11:43:11.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2023-10-06T11:43:11.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.523 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2023-10-06T11:43:11.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2023-10-06T11:43:11.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:11.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "7", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2023-10-06T11:43:11.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T11:43:11.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2023-10-06T11:43:11.538 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2400", 2023-10-06T11:43:11.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:11.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "7", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:11.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.542 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:11.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.545 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:11.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.550 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2023-10-06T11:43:11.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2023-10-06T11:43:11.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T11:43:11.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T11:43:11.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2023-10-06T11:43:11.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.565 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T11:43:11.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2023-10-06T11:43:11.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2023-10-06T11:43:11.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T11:43:11.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2023-10-06T11:43:11.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2023-10-06T11:43:11.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T11:43:11.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2023-10-06T11:43:11.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.583 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2023-10-06T11:43:11.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T11:43:11.585 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.586 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T11:43:11.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2023-10-06T11:43:11.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T11:43:11.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:11.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.604 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T11:43:11.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2023-10-06T11:43:11.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2023-10-06T11:43:11.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2023-10-06T11:43:11.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.627 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T11:43:11.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:11.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.631 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:11.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:11.634 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2023-10-06T11:43:11.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.638 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T11:43:11.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.950 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T11:43:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.952 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T11:43:11.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.958 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:11.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2023-10-06T11:43:11.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.967 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:11.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T11:43:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T11:43:11.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T11:43:12.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T11:43:12.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T11:43:12.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.117 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T11:43:12.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T11:43:12.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T11:43:12.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.127 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2023-10-06T11:43:12.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.139 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2023-10-06T11:43:12.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.152 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2023-10-06T11:43:12.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.155 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.158 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2023-10-06T11:43:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2023-10-06T11:43:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2023-10-06T11:43:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2023-10-06T11:43:12.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.289 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.292 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.298 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.304 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.305 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.308 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2023-10-06T11:43:12.310 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2023-10-06T11:43:12.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T11:43:12.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2023-10-06T11:43:12.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.317 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.320 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.329 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.334 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.335 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2023-10-06T11:43:12.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.341 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.344 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.349 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.350 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.351 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2023-10-06T11:43:12.362 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.363 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.366 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2023-10-06T11:43:12.369 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.374 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.375 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.378 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.383 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.384 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.387 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2023-10-06T11:43:12.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.391 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.394 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.395 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.398 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.403 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.405 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.406 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.408 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2023-10-06T11:43:12.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.411 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2023-10-06T11:43:12.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2023-10-06T11:43:12.418 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T11:43:12.428 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.429 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2023-10-06T11:43:12.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.437 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.444 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.449 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "drive_group_interval": { 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "drive_group_interval", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300.0", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.456 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2023-10-06T11:43:12.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2023-10-06T11:43:12.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.470 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.471 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2023-10-06T11:43:12.473 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2023-10-06T11:43:12.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2023-10-06T11:43:12.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2023-10-06T11:43:12.482 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.485 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.486 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.495 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.498 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.503 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.506 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.507 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.510 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.512 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2023-10-06T11:43:12.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2023-10-06T11:43:12.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2023-10-06T11:43:12.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.524 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.534 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.543 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.544 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.545 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.546 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.550 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.553 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.555 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.558 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.561 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.563 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2023-10-06T11:43:12.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2023-10-06T11:43:12.567 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "restful" 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.569 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.570 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.573 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2023-10-06T11:43:12.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2023-10-06T11:43:12.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.577 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2023-10-06T11:43:12.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2023-10-06T11:43:12.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.584 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.586 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.587 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "7", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T11:43:12.588 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-06T11:43:12.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2023-10-06T11:43:12.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2400", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2023-10-06T11:43:12.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "7", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 or 7 = Sunday, 1 = Monday, etc.", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.595 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.598 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-06T11:43:12.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap" 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.603 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.606 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:12.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.608 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2023-10-06T11:43:12.610 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-06T11:43:12.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.614 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2023-10-06T11:43:12.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.618 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-06T11:43:12.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2023-10-06T11:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.624 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.1.5", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-06T11:43:12.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2023-10-06T11:43:12.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2023-10-06T11:43:12.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.637 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.642 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.644 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2023-10-06T11:43:12.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-06T11:43:12.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.648 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2023-10-06T11:43:12.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.656 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2023-10-06T11:43:12.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.658 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-06T11:43:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2023-10-06T11:43:12.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.670 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.671 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.673 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.674 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-06T11:43:12.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-06T11:43:12.679 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-06T11:43:12.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.688 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.690 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-06T11:43:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.693 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.694 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.696 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.697 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2023-10-06T11:43:12.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.700 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-06T11:43:12.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.705 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.712 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-06T11:43:12.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.716 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-06T11:43:12.717 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-06T11:43:12.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-06T11:43:12.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.729 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2023-10-06T11:43:12.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.734 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.738 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.740 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.741 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.743 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2023-10-06T11:43:12.748 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.750 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.751 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2023-10-06T11:43:12.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2023-10-06T11:43:12.755 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.757 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.760 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2023-10-06T11:43:12.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.763 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2023-10-06T11:43:12.765 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2023-10-06T11:43:12.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.771 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.772 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.775 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.777 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.780 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2023-10-06T11:43:12.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-06T11:43:12.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.787 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.791 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2023-10-06T11:43:12.794 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.797 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2023-10-06T11:43:12.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2023-10-06T11:43:12.799 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.803 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.804 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.807 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.810 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.811 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.815 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2023-10-06T11:43:12.821 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.823 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.824 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.827 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.830 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.832 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.837 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.839 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.840 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.843 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.851 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.856 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.857 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.859 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.862 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.865 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.870 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.880 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2023-10-06T11:43:12.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2023-10-06T11:43:12.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.889 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.890 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.895 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.900 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.901 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.902 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2023-10-06T11:43:12.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.905 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.906 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.910 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2023-10-06T11:43:12.911 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2023-10-06T11:43:12.913 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.914 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.918 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.921 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2023-10-06T11:43:12.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.927 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "drive_group_interval": { 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "drive_group_interval", 2023-10-06T11:43:12.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300.0", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.934 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.937 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2023-10-06T11:43:12.939 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.940 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.945 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.947 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.948 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.950 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2023-10-06T11:43:12.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2023-10-06T11:43:12.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2023-10-06T11:43:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.964 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.968 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.971 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.972 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.974 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.978 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.983 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.987 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2023-10-06T11:43:12.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.993 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.995 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2023-10-06T11:43:12.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:12.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:13.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2023-10-06T11:43:13.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.007 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.010 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.013 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2023-10-06T11:43:13.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.015 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.023 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.030 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2023-10-06T11:43:13.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.035 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2023-10-06T11:43:13.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2023-10-06T11:43:13.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "services": {}, 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2023-10-06T11:43:13.044 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2023-10-06T11:43:13.045 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2023-10-06T11:43:13.046 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 0, 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3872668270 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2744304064 2023-10-06T11:43:13.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1329914895 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2310367126 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:13.050 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 978963432 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2707572356 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.177:0", 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3839067472 2023-10-06T11:43:13.052 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-10-06T11:43:13.053 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-10-06T11:43:13.053 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 msgr2=0x7fb8b4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 0x7fb8b4069590 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8007fd0 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 msgr2=0x7fb8c80c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80cf8c0 tx=0x7fb8c000b040 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 shutdown_connections 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8c80c62d0 0x7fb8c80c6750 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fb8b40670e0 0x7fb8b4069590 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8b0004810 0x7fb8c816e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 --2- 172.21.15.16:0/3276107924 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8c8154300 0x7fb8c8154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 >> 172.21.15.16:0/3276107924 conn(0x7fb8c8099b40 msgr2=0x7fb8c806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 shutdown_connections 2023-10-06T11:43:13.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:10.864+0000 7fb8d0640700 1 -- 172.21.15.16:0/3276107924 wait complete. 2023-10-06T11:43:13.059 DEBUG:tasks.check_counter:Getting stats from x 2023-10-06T11:43:13.059 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mgr.x.asok perf dump 2023-10-06T11:43:13.187 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 573, 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 367, 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 4084912, 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 2334620, 2023-10-06T11:43:13.188 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 77, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 21, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.109214763, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.022943662, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.087840028, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000096328, 2023-10-06T11:43:13.189 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 358, 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.027855882, 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000077809 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 67, 2023-10-06T11:43:13.190 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000008039, 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000119 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 483, 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 356, 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 2005984, 2023-10-06T11:43:13.191 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 2639292, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 76, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 21, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.052769982, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.009804282, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.044079804, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.002666127, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 337, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.212847544, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000631595 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.192 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 282, 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000030382, 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000107 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.193 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 329, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 144, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 2302896, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 586880, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 166, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 19, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.085730670, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.005404104, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.086603721, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000047432, 2023-10-06T11:43:13.194 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 130, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.012456192, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000095816 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 138, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000014049, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000101 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2023-10-06T11:43:13.195 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 0, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "client": { 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": { 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "lat": { 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.196 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "wrlat": { 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "rdlat": { 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.197 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "fsync": { 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "mdavg": 0.000000000, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "mdsqsum": 0, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "mdops": 0, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "readavg": 0.000000000, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "readsqsum": 0, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0, 2023-10-06T11:43:13.198 INFO:teuthology.orchestra.run.smithi177.stdout: "writeavg": 0.000000000, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "writesqsum": 0, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "client-0x555d6c52a8c0": { 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": { 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 70, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.092066311, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001315233 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "lat": { 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 70, 2023-10-06T11:43:13.199 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.092066311, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001315233 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "wrlat": { 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 18, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000568999, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000031611 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "rdlat": { 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 18, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000501814, 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000027878 2023-10-06T11:43:13.200 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "fsync": { 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.010799219, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003599739 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "mdavg": 0.001315215, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "mdsqsum": 120151909233799, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "mdops": 70, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "readavg": 0.000027874, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "readsqsum": 2729603037, 2023-10-06T11:43:13.201 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 18, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "writeavg": 0.000031607, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "writesqsum": 4626081256, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 18 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "client-0x555d6c52af00": { 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": { 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 54, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.084502758, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001564865 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "lat": { 2023-10-06T11:43:13.202 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 54, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.084502758, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001564865 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "wrlat": { 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "rdlat": { 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.203 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "fsync": { 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "mdavg": 0.001564853, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "mdsqsum": 175455486616453, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "mdops": 54, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "readavg": 0.000000000, 2023-10-06T11:43:13.204 INFO:teuthology.orchestra.run.smithi177.stdout: "readsqsum": 0, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "writeavg": 0.000000000, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "writesqsum": 0, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "client-0x555d6c52b5e0": { 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": { 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 21, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.373706944, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.017795568 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "lat": { 2023-10-06T11:43:13.205 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 21, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.373706944, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.017795568 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "wrlat": { 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "rdlat": { 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.206 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "fsync": { 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "mdavg": 0.017795562, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "mdsqsum": 122196306557372208, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "mdops": 21, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "readavg": 0.000000000, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "readsqsum": 0, 2023-10-06T11:43:13.207 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "writeavg": 0.000000000, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "writesqsum": 0, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "rdops": 0 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-Mgr": { 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.042485168, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.021242584 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.208 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MgrStandby": { 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-balancer": { 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.209 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-cmd_finisher": { 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 13, 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000747505, 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000057500 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-crash": { 2023-10-06T11:43:13.210 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-devicehealth": { 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.211 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-iostat": { 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-nfs": { 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.212 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-orchestrator": { 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.213 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-pg_autoscaler": { 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-progress": { 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.214 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-rbd_support": { 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.215 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-restful": { 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 13, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.002879010, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000221462 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-status": { 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.216 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-telemetry": { 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-volumes": { 2023-10-06T11:43:13.217 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 9, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.152696582, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.016966286 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "libcephsqlite_striper": { 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "update_metadata": 8, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "update_allocated": 2, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "update_size": 6, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "update_version": 8, 2023-10-06T11:43:13.218 INFO:teuthology.orchestra.run.smithi177.stdout: "shrink": 0, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "shrink_bytes": 0, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "lock": 11, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "unlock": 11 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "libcephsqlite_vfs": { 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "op_open": { 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.003999892, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001999946 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "op_delete": { 2023-10-06T11:43:13.219 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "op_access": { 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "op_fullpathname": { 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000999974, 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000999974 2023-10-06T11:43:13.220 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "op_currenttime": { 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 4, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_close": { 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_read": { 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 20, 2023-10-06T11:43:13.221 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.005999841, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000299992 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_write": { 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 85, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.004999868, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000058821 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_truncate": { 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.222 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_sync": { 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 24, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.081997823, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003416575 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_filesize": { 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 20, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_lock": { 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 27, 2023-10-06T11:43:13.223 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.018999493, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000703684 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_unlock": { 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 19, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.013999628, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000736822 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_checkreservedlock": { 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 9, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.224 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_filecontrol": { 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 26, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_sectorsize": { 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "opf_devicecharacteristics": { 2023-10-06T11:43:13.225 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 30, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2023-10-06T11:43:13.226 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Buffer_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Buffer_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Extent_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Extent_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Blob_bytes": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_Blob_items": 0, 2023-10-06T11:43:13.227 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_SharedBlob_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_SharedBlob_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2023-10-06T11:43:13.228 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 1597126, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 339, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 1496, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 17, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2023-10-06T11:43:13.229 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 49560, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 744, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 22808, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 364, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2023-10-06T11:43:13.230 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "mgr": { 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_hit": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_miss": 0 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "objectcacher-libcephfs": { 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_hit": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_miss": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_hit": 0, 2023-10-06T11:43:13.231 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_miss": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "data_read": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "data_written": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "data_flushed": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "data_overwritten_while_flushing": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "write_ops_blocked": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "write_bytes_blocked": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "write_time_blocked": 0.000000000 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "objectcacher-libcephfs-0x555d6c52a3c0": { 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_hit": 9, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_miss": 0, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_hit": 1431, 2023-10-06T11:43:13.232 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_miss": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "data_read": 1431, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "data_written": 477, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "data_flushed": 477, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "data_overwritten_while_flushing": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "write_ops_blocked": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "write_bytes_blocked": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "write_time_blocked": 0.000000000 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "objectcacher-libcephfs-0x555d6c52b040": { 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_hit": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_miss": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_hit": 0, 2023-10-06T11:43:13.233 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_miss": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "data_read": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "data_written": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "data_flushed": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "data_overwritten_while_flushing": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "write_ops_blocked": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "write_bytes_blocked": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "write_time_blocked": 0.000000000 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "objectcacher-libcephfs-0x555d6c52b720": { 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_hit": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_ops_miss": 0, 2023-10-06T11:43:13.234 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_hit": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "cache_bytes_miss": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "data_read": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "data_written": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "data_flushed": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "data_overwritten_while_flushing": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "write_ops_blocked": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "write_bytes_blocked": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "write_time_blocked": 0.000000000 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.235 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.236 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.237 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.238 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.239 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 8, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 35, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 33, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 8, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 8, 2023-10-06T11:43:13.240 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c3ca320": { 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 163, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 2434738, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.241 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 163, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 163, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 307 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 163, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 46, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 117, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.242 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 2, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 20, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 85, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.243 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 104, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 52, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 22, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 22, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.244 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.245 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 17, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 7, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 1, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 1, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.246 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c3ca6e0": { 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 0, 2023-10-06T11:43:13.247 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.248 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.249 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.250 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.251 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.252 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c3ca960": { 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 2, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 32, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 2, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2023-10-06T11:43:13.253 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 2, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 2, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.254 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.255 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.256 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.257 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 29, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 19, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 2, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 2, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c3cb7c0": { 2023-10-06T11:43:13.258 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 1, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 39, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 1, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 1 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.259 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 1, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 1, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.261 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.262 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 1, 2023-10-06T11:43:13.263 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.264 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 1, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 1, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.265 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c52a320": { 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 3, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 477, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 3, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3, 2023-10-06T11:43:13.266 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 3, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 3, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 3, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.267 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.268 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.269 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.270 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 3, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 3, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c52afa0": { 2023-10-06T11:43:13.271 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2023-10-06T11:43:13.272 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.273 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.274 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.275 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.276 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter-0x555d6c52b680": { 2023-10-06T11:43:13.277 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.278 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-10-06T11:43:13.279 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-10-06T11:43:13.280 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-10-06T11:43:13.281 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-10-06T11:43:13.282 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_client_bytes": { 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 134217728, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.283 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.284 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_client_messages": { 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 512, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.285 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_mds_bytes": { 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 134217728, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.286 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.287 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_mds_messsages": { 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 128, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.288 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_mon_bytes": { 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 134217728, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.289 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 502, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 5448002, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 502, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 502, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 5448002, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.290 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_mon_messsages": { 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 128, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 502, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 502, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 502, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 502, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 502, 2023-10-06T11:43:13.291 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_osd_bytes": { 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 536870912, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.292 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.293 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-mgr_osd_messsages": { 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 8192, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.294 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-client": { 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 173, 2023-10-06T11:43:13.295 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 63657, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 173, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 173, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 63657, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.296 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-client-0x555d6c52a780": { 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 41, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 21961, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 41, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 41, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 21961, 2023-10-06T11:43:13.297 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-client-0x555d6c52aaa0": { 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 84, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 43156, 2023-10-06T11:43:13.298 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 84, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 84, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 43156, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mgr": { 2023-10-06T11:43:13.299 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 339, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 811498, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 339, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 339, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 811498, 2023-10-06T11:43:13.300 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mgr-0x555d68919720": { 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 502, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 5448002, 2023-10-06T11:43:13.301 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 502, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 502, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 5448002, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.302 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-radosclient": { 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 177, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 512703, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 177, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 177, 2023-10-06T11:43:13.303 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 512703, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x555d6c3ca8c0": { 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 11, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 468603, 2023-10-06T11:43:13.304 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 11, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 11, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 468603, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.305 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x555d6c3cab40": { 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 16, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 483379, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 16, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 16, 2023-10-06T11:43:13.306 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 483379, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-radosclient-0x555d6c3cabe0": { 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.307 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 10, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 345531, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 10, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 10, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 345531, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.308 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c3ca1e0": { 2023-10-06T11:43:13.310 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 163, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2433049, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 163, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 163, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2433049, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.311 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c3ca820": { 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.312 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c3caaa0": { 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.313 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.314 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c3cac80": { 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1, 2023-10-06T11:43:13.315 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c52a500": { 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.316 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 3, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 477, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 477, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.317 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c52ab40": { 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.318 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes-0x555d6c52b860": { 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.319 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.320 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.321 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c3ca280": { 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 163, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 163, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 163, 2023-10-06T11:43:13.322 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 163, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 163, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c3ca780": { 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.323 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.324 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c3caa00": { 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 2, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 2, 2023-10-06T11:43:13.325 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 2, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c3cad20": { 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.326 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 1, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.327 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c52a460": { 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 3, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 3, 2023-10-06T11:43:13.328 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 3, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c52abe0": { 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.329 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.330 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops-0x555d6c52b7c0": { 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-10-06T11:43:13.331 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T11:43:13.332 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T11:43:13.334 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.complete_latency.avgcount=9 2023-10-06T11:43:13.334 INFO:tasks.check_counter:Daemon mgr.x finisher-volumes.queue_len=0 2023-10-06T11:43:13.334 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T11:43:13.343 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T11:43:13.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.343 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T11:43:13.359 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T11:43:13.359 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T11:43:13.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T11:43:13.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.431 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T11:43:13.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.513 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-10-06T11:43:13.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.606 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-10-06T11:43:13.625 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T11:43:13.626 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T11:43:13.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T11:43:13.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T11:43:13.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T11:43:13.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-10-06T11:43:13.844 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T11:43:13.854 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T11:43:13.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:13.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.977+0000 7fac0856a700 1 Processor -- start 2023-10-06T11:43:13.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.977+0000 7fac0856a700 1 -- start start 2023-10-06T11:43:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac0856a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac000b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac0856a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0006e7c0 con 0x7fac0015b0c0 2023-10-06T11:43:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac0856a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0006dbb0 con 0x7fac0015b530 2023-10-06T11:43:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac0856a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0006d9d0 con 0x7fac00153e60 2023-10-06T11:43:13.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac05b05700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac0015b0c0 0x7fac0006ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42832/0 (socket says 172.21.15.16:42832) 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac05b05700 1 -- 172.21.15.16:0/3322392195 learned_addr learned my addr 172.21.15.16:0/3322392195 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac06306700 1 --2- 172.21.15.16:0/3322392195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac000b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882198131 0 0) 0x7fac0006e7c0 con 0x7fac0015b0c0 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.978+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabec003570 con 0x7fac0015b0c0 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12652758 0 0) 0x7fac0006dbb0 con 0x7fac0015b530 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac0006e7c0 con 0x7fac0015b530 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac06306700 1 -- 172.21.15.16:0/3322392195 >> v1:172.21.15.177:6789/0 conn(0x7fac0015b530 legacy=0x7fac000ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1967950442 0 0) 0x7fabec003570 con 0x7fac0015b0c0 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac06306700 1 -- 172.21.15.16:0/3322392195 >> v1:172.21.15.16:6789/0 conn(0x7fac0015b0c0 legacy=0x7fac0006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac06306700 1 -- 172.21.15.16:0/3322392195 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0006de90 con 0x7fac00153e60 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac06306700 1 --2- 172.21.15.16:0/3322392195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac000b0890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009910 tx=0x7fabfc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46fa5b4da1185c4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:13.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0001470 con 0x7fac00153e60 2023-10-06T11:43:13.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc011420 con 0x7fac00153e60 2023-10-06T11:43:13.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.979+0000 7fac05304700 1 -- 172.21.15.16:0/3322392195 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc0117a0 con 0x7fac00153e60 2023-10-06T11:43:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 -- 172.21.15.16:0/3322392195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 msgr2=0x7fac000b0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 --2- 172.21.15.16:0/3322392195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac000b0890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009910 tx=0x7fabfc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 -- 172.21.15.16:0/3322392195 shutdown_connections 2023-10-06T11:43:13.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 --2- 172.21.15.16:0/3322392195 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac000b0890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 -- 172.21.15.16:0/3322392195 >> 172.21.15.16:0/3322392195 conn(0x7fac0006c410 msgr2=0x7fac0006c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 -- 172.21.15.16:0/3322392195 shutdown_connections 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.980+0000 7fac0856a700 1 -- 172.21.15.16:0/3322392195 wait complete. 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.981+0000 7fac0856a700 1 Processor -- start 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.981+0000 7fac0856a700 1 -- start start 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 0x7fac00152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00157fe0 con 0x7fac0015b530 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac001573d0 con 0x7fac0015b0c0 2023-10-06T11:43:13.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac001576b0 con 0x7fac00153e60 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06306700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac05b05700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 0x7fac00152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43646/0 (socket says 172.21.15.16:43646) 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06306700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:43876/0 (socket says 172.21.15.16:43876) 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 -- 172.21.15.16:0/3113911458 learned_addr learned my addr 172.21.15.16:0/3113911458 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 -- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 msgr2=0x7fac0015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 -- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 msgr2=0x7fac00152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 0x7fac00152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:13.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 -- 172.21.15.16:0/3113911458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009420 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06306700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac05b05700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 0x7fac00152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac06b07700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabf800c910 tx=0x7fabf80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8012630 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf80127b0 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac00157990 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf8012b70 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.982+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0015de10 con 0x7fac0015b530 2023-10-06T11:43:13.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.983+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac00158190 con 0x7fac0015b530 2023-10-06T11:43:13.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.984+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fabf801b400 con 0x7fac0015b530 2023-10-06T11:43:13.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.984+0000 7fabf67fc700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 0x7fabe4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:13.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.984+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fabf8084930 con 0x7fac0015b530 2023-10-06T11:43:13.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.984+0000 7fac06306700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 0x7fabe4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:13.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.985+0000 7fac06306700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 0x7fabe4069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabfc003080 tx=0x7fabfc016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:13.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:13.989+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac00158190 con 0x7fac0015b530 2023-10-06T11:43:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.185+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac0004c380 con 0x7fac0015b530 2023-10-06T11:43:14.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.186+0000 7fabf67fc700 1 -- 172.21.15.16:0/3113911458 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13779 (secure 0 0 0) 0x7fabf8051ac0 con 0x7fac0015b530 2023-10-06T11:43:14.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:14.188 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:43:07.918376+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1696671501":"2023-10-07T11:43:04.756021+0000","172.21.15.16:6833/1696671501":"2023-10-07T11:43:04.756021+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.188+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 msgr2=0x7fabe4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.188+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 0x7fabe4069590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fabfc003080 tx=0x7fabfc016000 comp rx=0 tx=0).stop 2023-10-06T11:43:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.188+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 msgr2=0x7fac000b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.188+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fabf800c910 tx=0x7fabf80027a0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 shutdown_connections 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac0015b530 0x7fac000b3520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fabe40670e0 0x7fabe4069590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fac00153e60 0x7fac0015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 --2- 172.21.15.16:0/3113911458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac0015b0c0 0x7fac00152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 >> 172.21.15.16:0/3113911458 conn(0x7fac0006c410 msgr2=0x7fac000ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 shutdown_connections 2023-10-06T11:43:14.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.189+0000 7fac0856a700 1 -- 172.21.15.16:0/3113911458 wait complete. 2023-10-06T11:43:14.202 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T11:43:14.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:43:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 Processor -- start 2023-10-06T11:43:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 -- start start 2023-10-06T11:43:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40cc15cc20 0x7f40cc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40cc0cd6a0 0x7f40cc0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0cdb10 con 0x7f40cc0cf280 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0b3110 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.321+0000 7f40d1744700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0ccb20 con 0x7f40cc15cc20 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40cc0cd6a0 0x7f40cc0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f40cc0cd6a0 0x7f40cc0ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43652/0 (socket says 172.21.15.16:43652) 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 -- 172.21.15.16:0/2794953332 learned_addr learned my addr 172.21.15.16:0/2794953332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40caffd700 1 --2- 172.21.15.16:0/2794953332 >> v2:172.21.15.177:3300/0 conn(0x7f40cc15cc20 0x7f40cc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 -- 172.21.15.16:0/2794953332 >> v2:172.21.15.177:3300/0 conn(0x7f40cc15cc20 msgr2=0x7f40cc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 --2- 172.21.15.16:0/2794953332 >> v2:172.21.15.177:3300/0 conn(0x7f40cc15cc20 0x7f40cc0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 -- 172.21.15.16:0/2794953332 >> v1:172.21.15.177:6789/0 conn(0x7f40cc0cf280 legacy=0x7f40cc15d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 -- 172.21.15.16:0/2794953332 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0cbf10 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40ca7fc700 1 --2- 172.21.15.16:0/2794953332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0ced40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f40cc089130 tx=0x7f40b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334da1088dd89de1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40c9ffb700 1 -- 172.21.15.16:0/2794953332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4009e70 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40c9ffb700 1 -- 172.21.15.16:0/2794953332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400b860 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.322+0000 7f40c9ffb700 1 -- 172.21.15.16:0/2794953332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400bc20 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 -- 172.21.15.16:0/2794953332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 msgr2=0x7f40cc0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 --2- 172.21.15.16:0/2794953332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0ced40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f40cc089130 tx=0x7f40b4009250 comp rx=0 tx=0).stop 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 -- 172.21.15.16:0/2794953332 shutdown_connections 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 --2- 172.21.15.16:0/2794953332 >> v2:172.21.15.177:3300/0 conn(0x7f40cc15cc20 0x7f40cc0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 --2- 172.21.15.16:0/2794953332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0ced40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 -- 172.21.15.16:0/2794953332 >> 172.21.15.16:0/2794953332 conn(0x7f40cc099b40 msgr2=0x7f40cc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 -- 172.21.15.16:0/2794953332 shutdown_connections 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.323+0000 7f40d1744700 1 -- 172.21.15.16:0/2794953332 wait complete. 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.324+0000 7f40d1744700 1 Processor -- start 2023-10-06T11:43:14.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.324+0000 7f40d1744700 1 -- start start 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40cc0cf280 0x7f40cc152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40cc15cc20 0x7f40cc153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b38e0 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0c9b90 con 0x7f40cc15cc20 2023-10-06T11:43:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40d1744700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0c9e70 con 0x7f40cc0cf280 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43668/0 (socket says 172.21.15.16:43668) 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 -- 172.21.15.16:0/3509596306 learned_addr learned my addr 172.21.15.16:0/3509596306 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40cb7fe700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40cc15cc20 0x7f40cc153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40ca7fc700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40cc0cf280 0x7f40cc152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 -- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40cc0cf280 msgr2=0x7f40cc152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40cc0cf280 0x7f40cc152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 -- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40cc15cc20 msgr2=0x7f40cc153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40cc15cc20 0x7f40cc153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 -- 172.21.15.16:0/3509596306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40094f0 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.325+0000 7f40caffd700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00ec80 tx=0x7f40bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.326+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00c850 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.326+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40bc00c9b0 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.326+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc0ca150 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.326+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40bc00cd50 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.326+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc0c6fb0 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.327+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc04c380 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.327+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f40bc013680 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.327+0000 7f40c2ffd700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 0x7f40ac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.327+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f40bc0848c0 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.328+0000 7f40ca7fc700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 0x7f40ac069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.328+0000 7f40ca7fc700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 0x7f40ac069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40cc089130 tx=0x7f40b4017000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.332+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40bc051a50 con 0x7f40cc0cd6a0 2023-10-06T11:43:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.523+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40cc0b3a20 con 0x7f40ac0670e0 2023-10-06T11:43:14.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.527+0000 7f40c2ffd700 1 -- 172.21.15.16:0/3509596306 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72445 (secure 0 0 0) 0x7f40b4020040 con 0x7f40ac0670e0 2023-10-06T11:43:14.528 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 msgr2=0x7f40ac069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 0x7f40ac069590 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40cc089130 tx=0x7f40b4017000 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 msgr2=0x7f40cc0c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40bc00ec80 tx=0x7f40bc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 shutdown_connections 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f40cc0cd6a0 0x7f40cc0c6a70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f40ac0670e0 0x7f40ac069590 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40cc15cc20 0x7f40cc153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 --2- 172.21.15.16:0/3509596306 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40cc0cf280 0x7f40cc152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 >> 172.21.15.16:0/3509596306 conn(0x7f40cc099b40 msgr2=0x7f40cc0c5620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 shutdown_connections 2023-10-06T11:43:14.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.529+0000 7f40d1744700 1 -- 172.21.15.16:0/3509596306 wait complete. 2023-10-06T11:43:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:43:14.542 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T11:43:14.077620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386684,"kb_used_data":4284,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588036,"statfs":{"total":773094113280,"available":770650148864,"internally_reserved":0,"allocated":4386816,"data_stored":2484606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-143061,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-101,"num_write_kb":-178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001433"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171547+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:43:07.171547+0000","last_peered":"2023-10-06T11:43:07.171547+0000","last_clean":"2023-10-06T11:43:07.171547+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:43:07.171547+0000","last_undegraded":"2023-10-06T11:43:07.171547+0000","last_fullsized":"2023-10-06T11:43:07.171547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921594+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:43:06.921594+0000","last_peered":"2023-10-06T11:43:06.921594+0000","last_clean":"2023-10-06T11:43:06.921594+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:43:06.921594+0000","last_undegraded":"2023-10-06T11:43:06.921594+0000","last_fullsized":"2023-10-06T11:43:06.921594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171541+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:43:07.171541+0000","last_peered":"2023-10-06T11:43:07.171541+0000","last_clean":"2023-10-06T11:43:07.171541+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:43:07.171541+0000","last_undegraded":"2023-10-06T11:43:07.171541+0000","last_fullsized":"2023-10-06T11:43:07.171541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921112+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:43:06.921112+0000","last_peered":"2023-10-06T11:43:06.921112+0000","last_clean":"2023-10-06T11:43:06.921112+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:43:06.921112+0000","last_undegraded":"2023-10-06T11:43:06.921112+0000","last_fullsized":"2023-10-06T11:43:06.921112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921356+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:43:06.921356+0000","last_peered":"2023-10-06T11:43:06.921356+0000","last_clean":"2023-10-06T11:43:06.921356+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:43:06.921356+0000","last_undegraded":"2023-10-06T11:43:06.921356+0000","last_fullsized":"2023-10-06T11:43:06.921356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921017+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:43:06.921017+0000","last_peered":"2023-10-06T11:43:06.921017+0000","last_clean":"2023-10-06T11:43:06.921017+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:43:06.921017+0000","last_undegraded":"2023-10-06T11:43:06.921017+0000","last_fullsized":"2023-10-06T11:43:06.921017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.170606+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:43:07.170606+0000","last_peered":"2023-10-06T11:43:07.170606+0000","last_clean":"2023-10-06T11:43:07.170606+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:43:07.170606+0000","last_undegraded":"2023-10-06T11:43:07.170606+0000","last_fullsized":"2023-10-06T11:43:07.170606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.923754+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:43:06.923754+0000","last_peered":"2023-10-06T11:43:06.923754+0000","last_clean":"2023-10-06T11:43:06.923754+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:43:06.923754+0000","last_undegraded":"2023-10-06T11:43:06.923754+0000","last_fullsized":"2023-10-06T11:43:06.923754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.924777+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:43:06.924777+0000","last_peered":"2023-10-06T11:43:06.924777+0000","last_clean":"2023-10-06T11:43:06.924777+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:43:06.924777+0000","last_undegraded":"2023-10-06T11:43:06.924777+0000","last_fullsized":"2023-10-06T11:43:06.924777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.75},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.86099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.624},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.748},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.497}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.378},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.26500000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":157943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":158921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.253},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.31},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.372},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":0.5}]},{"osd":1,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.7889999999999997,"5min":6.7889999999999997,"15min":6.7889999999999997},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:43:14.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:43:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.662+0000 7fcec6704700 1 Processor -- start 2023-10-06T11:43:14.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.662+0000 7fcec6704700 1 -- start start 2023-10-06T11:43:14.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec6704700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 0x7fcec015d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec6704700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcec0159790 0x7fcec015d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec6704700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec015a650 con 0x7fcec00b53f0 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec6704700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00b5f60 con 0x7fcec0159790 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec6704700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b61c0 con 0x7fcec01593c0 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec4f01700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcec0159790 0x7fcec015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec4f01700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcec0159790 0x7fcec015d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43682/0 (socket says 172.21.15.16:43682) 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec5f03700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcec00b53f0 0x7fcec00b5850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42848/0 (socket says 172.21.15.16:42848) 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec4f01700 1 -- 172.21.15.16:0/2653791118 learned_addr learned my addr 172.21.15.16:0/2653791118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.663+0000 7fcec5702700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 0x7fcec015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec4f01700 1 -- 172.21.15.16:0/2653791118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 msgr2=0x7fcec015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:14.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec4f01700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 0x7fcec015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec4f01700 1 -- 172.21.15.16:0/2653791118 >> v1:172.21.15.16:6789/0 conn(0x7fcec00b53f0 legacy=0x7fcec00b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec4f01700 1 -- 172.21.15.16:0/2653791118 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0159d20 con 0x7fcec0159790 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec5702700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 0x7fcec015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec4f01700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec0159790 0x7fcec015d750 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009990 tx=0x7fceb4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd6d6df2411f133 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcebffff700 1 -- 172.21.15.16:0/2653791118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4005c10 con 0x7fcec0159790 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcebffff700 1 -- 172.21.15.16:0/2653791118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4005d70 con 0x7fcec0159790 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcebffff700 1 -- 172.21.15.16:0/2653791118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400b880 con 0x7fcec0159790 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec6704700 1 -- 172.21.15.16:0/2653791118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec0159790 msgr2=0x7fcec015d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec6704700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec0159790 0x7fcec015d750 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009990 tx=0x7fceb4009220 comp rx=0 tx=0).stop 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec6704700 1 -- 172.21.15.16:0/2653791118 shutdown_connections 2023-10-06T11:43:14.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec6704700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec01593c0 0x7fcec015d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.664+0000 7fcec6704700 1 --2- 172.21.15.16:0/2653791118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec0159790 0x7fcec015d750 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.665+0000 7fcec6704700 1 -- 172.21.15.16:0/2653791118 >> 172.21.15.16:0/2653791118 conn(0x7fcec008a020 msgr2=0x7fcec0099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.665+0000 7fcec6704700 1 -- 172.21.15.16:0/2653791118 shutdown_connections 2023-10-06T11:43:14.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.665+0000 7fcec6704700 1 -- 172.21.15.16:0/2653791118 wait complete. 2023-10-06T11:43:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.666+0000 7fcec6704700 1 Processor -- start 2023-10-06T11:43:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.666+0000 7fcec6704700 1 -- start start 2023-10-06T11:43:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcec01593c0 0x7fcec0152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec0159790 0x7fcec0150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec014f970 con 0x7fcec00b53f0 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0151930 con 0x7fcec01593c0 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0151b80 con 0x7fcec0159790 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43688/0 (socket says 172.21.15.16:43688) 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 -- 172.21.15.16:0/951440273 learned_addr learned my addr 172.21.15.16:0/951440273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec4f01700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcec01593c0 0x7fcec0152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5f03700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec0159790 0x7fcec0150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 -- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec0159790 msgr2=0x7fcec0150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec0159790 0x7fcec0150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 -- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcec01593c0 msgr2=0x7fcec0152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcec01593c0 0x7fcec0152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 -- 172.21.15.16:0/951440273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb4009480 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec5702700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fceb800d220 tx=0x7fceb8009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb8015440 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb800f040 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec0151ec0 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80157d0 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.667+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec014d500 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.668+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec0152790 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.669+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fceb8015c00 con 0x7fcec00b53f0 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.669+0000 7fcebd7fa700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 0x7fceac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.669+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fceb8084cb0 con 0x7fcec00b53f0 2023-10-06T11:43:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.669+0000 7fcec4f01700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 0x7fceac069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.670+0000 7fcec4f01700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 0x7fceac069590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009260 tx=0x7fceb4016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:14.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.673+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fceb804e5a0 con 0x7fcec00b53f0 2023-10-06T11:43:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.862+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcec01522b0 con 0x7fceac0670e0 2023-10-06T11:43:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.865+0000 7fcebd7fa700 1 -- 172.21.15.16:0/951440273 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72445 (secure 0 0 0) 0x7fcec01522b0 con 0x7fceac0670e0 2023-10-06T11:43:14.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 msgr2=0x7fceac069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 0x7fceac069590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009260 tx=0x7fceb4016000 comp rx=0 tx=0).stop 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 msgr2=0x7fcec014cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fceb800d220 tx=0x7fceb8009ae0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 shutdown_connections 2023-10-06T11:43:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec00b53f0 0x7fcec014cfc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fceac0670e0 0x7fceac069590 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcec0159790 0x7fcec0150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 --2- 172.21.15.16:0/951440273 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcec01593c0 0x7fcec0152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 >> 172.21.15.16:0/951440273 conn(0x7fcec008a020 msgr2=0x7fcec0154e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 shutdown_connections 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:14.867+0000 7fcec6704700 1 -- 172.21.15.16:0/951440273 wait complete. 2023-10-06T11:43:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:43:14.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T11:43:14.077620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386684,"kb_used_data":4284,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588036,"statfs":{"total":773094113280,"available":770650148864,"internally_reserved":0,"allocated":4386816,"data_stored":2484606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-143061,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-101,"num_write_kb":-178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001433"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171547+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:43:07.171547+0000","last_peered":"2023-10-06T11:43:07.171547+0000","last_clean":"2023-10-06T11:43:07.171547+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:43:07.171547+0000","last_undegraded":"2023-10-06T11:43:07.171547+0000","last_fullsized":"2023-10-06T11:43:07.171547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921594+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:43:06.921594+0000","last_peered":"2023-10-06T11:43:06.921594+0000","last_clean":"2023-10-06T11:43:06.921594+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:43:06.921594+0000","last_undegraded":"2023-10-06T11:43:06.921594+0000","last_fullsized":"2023-10-06T11:43:06.921594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171541+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:43:07.171541+0000","last_peered":"2023-10-06T11:43:07.171541+0000","last_clean":"2023-10-06T11:43:07.171541+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:43:07.171541+0000","last_undegraded":"2023-10-06T11:43:07.171541+0000","last_fullsized":"2023-10-06T11:43:07.171541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921112+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:43:06.921112+0000","last_peered":"2023-10-06T11:43:06.921112+0000","last_clean":"2023-10-06T11:43:06.921112+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:43:06.921112+0000","last_undegraded":"2023-10-06T11:43:06.921112+0000","last_fullsized":"2023-10-06T11:43:06.921112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921356+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:43:06.921356+0000","last_peered":"2023-10-06T11:43:06.921356+0000","last_clean":"2023-10-06T11:43:06.921356+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:43:06.921356+0000","last_undegraded":"2023-10-06T11:43:06.921356+0000","last_fullsized":"2023-10-06T11:43:06.921356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921017+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:43:06.921017+0000","last_peered":"2023-10-06T11:43:06.921017+0000","last_clean":"2023-10-06T11:43:06.921017+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:43:06.921017+0000","last_undegraded":"2023-10-06T11:43:06.921017+0000","last_fullsized":"2023-10-06T11:43:06.921017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.170606+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:43:07.170606+0000","last_peered":"2023-10-06T11:43:07.170606+0000","last_clean":"2023-10-06T11:43:07.170606+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:43:07.170606+0000","last_undegraded":"2023-10-06T11:43:07.170606+0000","last_fullsized":"2023-10-06T11:43:07.170606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.923754+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:43:06.923754+0000","last_peered":"2023-10-06T11:43:06.923754+0000","last_clean":"2023-10-06T11:43:06.923754+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:43:06.923754+0000","last_undegraded":"2023-10-06T11:43:06.923754+0000","last_fullsized":"2023-10-06T11:43:06.923754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.924777+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:43:06.924777+0000","last_peered":"2023-10-06T11:43:06.924777+0000","last_clean":"2023-10-06T11:43:06.924777+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:43:06.924777+0000","last_undegraded":"2023-10-06T11:43:06.924777+0000","last_fullsized":"2023-10-06T11:43:06.924777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.75},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.86099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.624},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.748},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.497}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.378},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.26500000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":157943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":158921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.253},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.31},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.372},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":0.5}]},{"osd":1,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.7889999999999997,"5min":6.7889999999999997,"15min":6.7889999999999997},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:43:14.882 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T11:43:14.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:43:15.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.000+0000 7fe5da6b2700 1 Processor -- start 2023-10-06T11:43:15.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.000+0000 7fe5da6b2700 1 -- start start 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5da6b2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5d4156550 0x7fe5d4069270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5da6b2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe5d40b0ea0 0x7fe5d4156920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5da6b2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40b2210 con 0x7fe5d4156e60 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5da6b2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40b1600 con 0x7fe5d4156550 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5da6b2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40b18e0 con 0x7fe5d40b0ea0 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5d3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5d4156550 0x7fe5d4069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5d3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5d4156550 0x7fe5d4069270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43704/0 (socket says 172.21.15.16:43704) 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.001+0000 7fe5d3fff700 1 -- 172.21.15.16:0/1136272184 learned_addr learned my addr 172.21.15.16:0/1136272184 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d37fe700 1 --2- 172.21.15.16:0/1136272184 >> v2:172.21.15.177:3300/0 conn(0x7fe5d40b0ea0 0x7fe5d4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d3fff700 1 -- 172.21.15.16:0/1136272184 >> v2:172.21.15.177:3300/0 conn(0x7fe5d40b0ea0 msgr2=0x7fe5d4156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d3fff700 1 --2- 172.21.15.16:0/1136272184 >> v2:172.21.15.177:3300/0 conn(0x7fe5d40b0ea0 0x7fe5d4156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d3fff700 1 -- 172.21.15.16:0/1136272184 >> v1:172.21.15.177:6789/0 conn(0x7fe5d4156e60 legacy=0x7fe5d406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d3fff700 1 -- 172.21.15.16:0/1136272184 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40b1bc0 con 0x7fe5d4156550 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d3fff700 1 --2- 172.21.15.16:0/1136272184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156550 0x7fe5d4069270 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406bd20 tx=0x7fe5c800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e6e1a46e35b767 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d2ffd700 1 -- 172.21.15.16:0/1136272184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c800dc50 con 0x7fe5d4156550 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d2ffd700 1 -- 172.21.15.16:0/1136272184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c800ddb0 con 0x7fe5d4156550 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.002+0000 7fe5d2ffd700 1 -- 172.21.15.16:0/1136272184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80196d0 con 0x7fe5d4156550 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 -- 172.21.15.16:0/1136272184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156550 msgr2=0x7fe5d4069270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/1136272184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156550 0x7fe5d4069270 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406bd20 tx=0x7fe5c800b4f0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 -- 172.21.15.16:0/1136272184 shutdown_connections 2023-10-06T11:43:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/1136272184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156550 0x7fe5d4069270 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/1136272184 >> v2:172.21.15.177:3300/0 conn(0x7fe5d40b0ea0 0x7fe5d4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 -- 172.21.15.16:0/1136272184 >> 172.21.15.16:0/1136272184 conn(0x7fe5d40cd480 msgr2=0x7fe5d40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 -- 172.21.15.16:0/1136272184 shutdown_connections 2023-10-06T11:43:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.003+0000 7fe5da6b2700 1 -- 172.21.15.16:0/1136272184 wait complete. 2023-10-06T11:43:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.004+0000 7fe5da6b2700 1 Processor -- start 2023-10-06T11:43:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.004+0000 7fe5da6b2700 1 -- start start 2023-10-06T11:43:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d40b0ea0 0x7fe5d40c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d4156550 0x7fe5d41648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d416ed70 con 0x7fe5d4156e60 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c44f0 con 0x7fe5d40b0ea0 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40c47d0 con 0x7fe5d4156550 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43716/0 (socket says 172.21.15.16:43716) 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 -- 172.21.15.16:0/341325673 learned_addr learned my addr 172.21.15.16:0/341325673 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d3fff700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d40b0ea0 0x7fe5d40c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d37fe700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d4156550 0x7fe5d41648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 -- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d4156550 msgr2=0x7fe5d41648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d4156550 0x7fe5d41648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 -- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d40b0ea0 msgr2=0x7fe5d40c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d40b0ea0 0x7fe5d40c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 -- 172.21.15.16:0/341325673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c800b790 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5d8c4f700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c8e0 tx=0x7fe5cc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc012630 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc012790 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40c4ab0 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.005+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc012b50 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.006+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d40ae1e0 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.006+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d404c380 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.007+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fe5cc01a3a0 con 0x7fe5d4156e60 2023-10-06T11:43:15.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.007+0000 7fe5bbfff700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 0x7fe5c0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.007+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fe5cc084810 con 0x7fe5d4156e60 2023-10-06T11:43:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.007+0000 7fe5d3fff700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 0x7fe5c0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.008+0000 7fe5d3fff700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 0x7fe5c0069590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406bd20 tx=0x7fe5c801b000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.012+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5cc0519a0 con 0x7fe5d4156e60 2023-10-06T11:43:15.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.199+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5d40c4370 con 0x7fe5c00670e0 2023-10-06T11:43:15.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.202+0000 7fe5bbfff700 1 -- 172.21.15.16:0/341325673 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72445 (secure 0 0 0) 0x7fe5c8000c50 con 0x7fe5c00670e0 2023-10-06T11:43:15.204 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 msgr2=0x7fe5c0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 0x7fe5c0069590 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406bd20 tx=0x7fe5c801b000 comp rx=0 tx=0).stop 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 msgr2=0x7fe5d40adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c8e0 tx=0x7fe5cc001710 comp rx=0 tx=0).stop 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 shutdown_connections 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5d4156e60 0x7fe5d40adca0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fe5c00670e0 0x7fe5c0069590 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d40b0ea0 0x7fe5d40c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 --2- 172.21.15.16:0/341325673 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d4156550 0x7fe5d41648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 >> 172.21.15.16:0/341325673 conn(0x7fe5d40cd480 msgr2=0x7fe5d40cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 shutdown_connections 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.205+0000 7fe5da6b2700 1 -- 172.21.15.16:0/341325673 wait complete. 2023-10-06T11:43:15.207 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:43:15.218 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2023-10-06T11:43:14.077620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386684,"kb_used_data":4284,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588036,"statfs":{"total":773094113280,"available":770650148864,"internally_reserved":0,"allocated":4386816,"data_stored":2484606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-143061,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-101,"num_write_kb":-178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001433"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171547+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:43:07.171547+0000","last_peered":"2023-10-06T11:43:07.171547+0000","last_clean":"2023-10-06T11:43:07.171547+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:43:07.171547+0000","last_undegraded":"2023-10-06T11:43:07.171547+0000","last_fullsized":"2023-10-06T11:43:07.171547+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921594+0000","last_change":"2023-10-06T11:41:50.458178+0000","last_active":"2023-10-06T11:43:06.921594+0000","last_peered":"2023-10-06T11:43:06.921594+0000","last_clean":"2023-10-06T11:43:06.921594+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:43:06.921594+0000","last_undegraded":"2023-10-06T11:43:06.921594+0000","last_fullsized":"2023-10-06T11:43:06.921594+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.171541+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:43:07.171541+0000","last_peered":"2023-10-06T11:43:07.171541+0000","last_clean":"2023-10-06T11:43:07.171541+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:43:07.171541+0000","last_undegraded":"2023-10-06T11:43:07.171541+0000","last_fullsized":"2023-10-06T11:43:07.171541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921112+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:43:06.921112+0000","last_peered":"2023-10-06T11:43:06.921112+0000","last_clean":"2023-10-06T11:43:06.921112+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:43:06.921112+0000","last_undegraded":"2023-10-06T11:43:06.921112+0000","last_fullsized":"2023-10-06T11:43:06.921112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921356+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:43:06.921356+0000","last_peered":"2023-10-06T11:43:06.921356+0000","last_clean":"2023-10-06T11:43:06.921356+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:43:06.921356+0000","last_undegraded":"2023-10-06T11:43:06.921356+0000","last_fullsized":"2023-10-06T11:43:06.921356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921017+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:43:06.921017+0000","last_peered":"2023-10-06T11:43:06.921017+0000","last_clean":"2023-10-06T11:43:06.921017+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:43:06.921017+0000","last_undegraded":"2023-10-06T11:43:06.921017+0000","last_fullsized":"2023-10-06T11:43:06.921017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.170606+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:43:07.170606+0000","last_peered":"2023-10-06T11:43:07.170606+0000","last_clean":"2023-10-06T11:43:07.170606+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:43:07.170606+0000","last_undegraded":"2023-10-06T11:43:07.170606+0000","last_fullsized":"2023-10-06T11:43:07.170606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.923754+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:43:06.923754+0000","last_peered":"2023-10-06T11:43:06.923754+0000","last_clean":"2023-10-06T11:43:06.923754+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:43:06.923754+0000","last_undegraded":"2023-10-06T11:43:06.923754+0000","last_fullsized":"2023-10-06T11:43:06.923754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.924777+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:43:06.924777+0000","last_peered":"2023-10-06T11:43:06.924777+0000","last_clean":"2023-10-06T11:43:06.924777+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:43:06.924777+0000","last_undegraded":"2023-10-06T11:43:06.924777+0000","last_fullsized":"2023-10-06T11:43:06.924777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.75},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.86099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.624},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.748},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.432}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.47599999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.497}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.378},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35499999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.26500000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297664,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074176,"statfs":{"total":96636764160,"available":96331956224,"internally_reserved":0,"allocated":385024,"data_stored":157943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.749},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.55300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297668,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074172,"statfs":{"total":96636764160,"available":96331952128,"internally_reserved":0,"allocated":389120,"data_stored":158921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.29399999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.30199999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.50700000000000001}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.253},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.28999999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.31},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.28199999999999997}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.35099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.372},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":0.5}]},{"osd":1,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.7889999999999997,"5min":6.7889999999999997,"15min":6.7889999999999997},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.65900000000000003}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.435}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.79300000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:43:15.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T11:43:15.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.337+0000 7f3724e64700 1 Processor -- start 2023-10-06T11:43:15.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.337+0000 7f3724e64700 1 -- start start 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f3724e64700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3720156ef0 0x7f37201572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f3724e64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37200b1c60 0x7f372015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f3724e64700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37200b1680 con 0x7f372006d520 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f3724e64700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372015ac40 con 0x7f37200b1c60 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f3724e64700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372006f540 con 0x7f3720156ef0 2023-10-06T11:43:15.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.338+0000 7f371dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37200b1c60 0x7f372015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37200b1c60 0x7f372015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43724/0 (socket says 172.21.15.16:43724) 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 -- 172.21.15.16:0/644635123 learned_addr learned my addr 172.21.15.16:0/644635123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371e59c700 1 --2- 172.21.15.16:0/644635123 >> v2:172.21.15.177:3300/0 conn(0x7f3720156ef0 0x7f37201572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 -- 172.21.15.16:0/644635123 >> v2:172.21.15.177:3300/0 conn(0x7f3720156ef0 msgr2=0x7f37201572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 --2- 172.21.15.16:0/644635123 >> v2:172.21.15.177:3300/0 conn(0x7f3720156ef0 0x7f37201572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 -- 172.21.15.16:0/644635123 >> v1:172.21.15.177:6789/0 conn(0x7f372006d520 legacy=0x7f37200b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 -- 172.21.15.16:0/644635123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200b21b0 con 0x7f37200b1c60 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371dd9b700 1 --2- 172.21.15.16:0/644635123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37200b1c60 0x7f372015a700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f37200cf310 tx=0x7f3708009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac75b33214dcfa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371d59a700 1 -- 172.21.15.16:0/644635123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708009e70 con 0x7f37200b1c60 2023-10-06T11:43:15.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371d59a700 1 -- 172.21.15.16:0/644635123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370800b860 con 0x7f37200b1c60 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.339+0000 7f371d59a700 1 -- 172.21.15.16:0/644635123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370800bc20 con 0x7f37200b1c60 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 -- 172.21.15.16:0/644635123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37200b1c60 msgr2=0x7f372015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 --2- 172.21.15.16:0/644635123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37200b1c60 0x7f372015a700 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f37200cf310 tx=0x7f3708009250 comp rx=0 tx=0).stop 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 -- 172.21.15.16:0/644635123 shutdown_connections 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 --2- 172.21.15.16:0/644635123 >> v2:172.21.15.177:3300/0 conn(0x7f3720156ef0 0x7f37201572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 --2- 172.21.15.16:0/644635123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37200b1c60 0x7f372015a700 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 -- 172.21.15.16:0/644635123 >> 172.21.15.16:0/644635123 conn(0x7f3720067e60 msgr2=0x7f3720068260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 -- 172.21.15.16:0/644635123 shutdown_connections 2023-10-06T11:43:15.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.340+0000 7f3724e64700 1 -- 172.21.15.16:0/644635123 wait complete. 2023-10-06T11:43:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.341+0000 7f3724e64700 1 Processor -- start 2023-10-06T11:43:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.341+0000 7f3724e64700 1 -- start start 2023-10-06T11:43:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372006d520 0x7f3720154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37200b1c60 0x7f37200ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200af5a0 con 0x7f3720156ef0 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200af880 con 0x7f37200b1c60 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200afb60 con 0x7f372006d520 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43732/0 (socket says 172.21.15.16:43732) 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 -- 172.21.15.16:0/26203542 learned_addr learned my addr 172.21.15.16:0/26203542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371dd9b700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37200b1c60 0x7f37200ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371e59c700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372006d520 0x7f3720154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 -- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372006d520 msgr2=0x7f3720154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372006d520 0x7f3720154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 -- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37200b1c60 msgr2=0x7f37200ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37200b1c60 0x7f37200ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 -- 172.21.15.16:0/26203542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080094f0 con 0x7f3720156ef0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f371ed9d700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f371800c8e0 tx=0x7f3718001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718012630 con 0x7f3720156ef0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3718012790 con 0x7f3720156ef0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.342+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37200afe40 con 0x7f3720156ef0 2023-10-06T11:43:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.343+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718012b50 con 0x7f3720156ef0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.343+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37200af280 con 0x7f3720156ef0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.343+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37201699a0 con 0x7f3720156ef0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.344+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f371801a3a0 con 0x7f3720156ef0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.344+0000 7f37167fc700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 0x7f3700069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.344+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f37180848b0 con 0x7f3720156ef0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.345+0000 7f371e59c700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 0x7f3700069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.345+0000 7f371e59c700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 0x7f3700069590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d00 tx=0x7f3710009120 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:15.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.349+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3718051a40 con 0x7f3720156ef0 2023-10-06T11:43:15.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.543+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f372004c380 con 0x7f3720156ef0 2023-10-06T11:43:15.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.544+0000 7f37167fc700 1 -- 172.21.15.16:0/26203542 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13779 (secure 0 0 0) 0x7f3718055060 con 0x7f3720156ef0 2023-10-06T11:43:15.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:15.546 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"b7fe6d1b-a342-44e2-9c53-ba3d26f05e41","created":"2023-10-06T11:41:37.648065+0000","modified":"2023-10-06T11:43:07.918376+0000","last_up_change":"2023-10-06T11:41:48.441522+0000","last_in_change":"2023-10-06T11:41:40.096177+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T11:41:48.881552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T11:41:50.113038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"af0d6ca8-3aba-4727-abbc-b852e772b0b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":90625},{"type":"v1","addr":"172.21.15.16:6809","nonce":90625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":90625},{"type":"v1","addr":"172.21.15.16:6811","nonce":90625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":90625},{"type":"v1","addr":"172.21.15.16:6815","nonce":90625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":90625},{"type":"v1","addr":"172.21.15.16:6813","nonce":90625}]},"public_addr":"172.21.15.16:6809/90625","cluster_addr":"172.21.15.16:6811/90625","heartbeat_back_addr":"172.21.15.16:6815/90625","heartbeat_front_addr":"172.21.15.16:6813/90625","state":["exists","up"]},{"osd":1,"uuid":"d38f9116-9742-4c46-bd6e-20844ce41a41","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":90616},{"type":"v1","addr":"172.21.15.16:6801","nonce":90616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":90616},{"type":"v1","addr":"172.21.15.16:6803","nonce":90616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":90616},{"type":"v1","addr":"172.21.15.16:6807","nonce":90616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":90616},{"type":"v1","addr":"172.21.15.16:6805","nonce":90616}]},"public_addr":"172.21.15.16:6801/90616","cluster_addr":"172.21.15.16:6803/90616","heartbeat_back_addr":"172.21.15.16:6807/90616","heartbeat_front_addr":"172.21.15.16:6805/90616","state":["exists","up"]},{"osd":2,"uuid":"0afa886b-62c7-409b-907c-81d69b99b2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6819","nonce":90626},{"type":"v1","addr":"172.21.15.16:6821","nonce":90626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6823","nonce":90626},{"type":"v1","addr":"172.21.15.16:6825","nonce":90626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":90626},{"type":"v1","addr":"172.21.15.16:6831","nonce":90626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6827","nonce":90626},{"type":"v1","addr":"172.21.15.16:6828","nonce":90626}]},"public_addr":"172.21.15.16:6821/90626","cluster_addr":"172.21.15.16:6825/90626","heartbeat_back_addr":"172.21.15.16:6831/90626","heartbeat_front_addr":"172.21.15.16:6828/90626","state":["exists","up"]},{"osd":3,"uuid":"e28cf380-7e4a-4a73-870d-f37e2436099c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":90633},{"type":"v1","addr":"172.21.15.16:6817","nonce":90633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":90633},{"type":"v1","addr":"172.21.15.16:6820","nonce":90633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":90633},{"type":"v1","addr":"172.21.15.16:6829","nonce":90633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":90633},{"type":"v1","addr":"172.21.15.16:6824","nonce":90633}]},"public_addr":"172.21.15.16:6817/90633","cluster_addr":"172.21.15.16:6820/90633","heartbeat_back_addr":"172.21.15.16:6829/90633","heartbeat_front_addr":"172.21.15.16:6824/90633","state":["exists","up"]},{"osd":4,"uuid":"a175cb92-fb78-4cbb-91f5-5df435916564","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":90842},{"type":"v1","addr":"172.21.15.177:6809","nonce":90842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":90842},{"type":"v1","addr":"172.21.15.177:6811","nonce":90842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":90842},{"type":"v1","addr":"172.21.15.177:6815","nonce":90842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":90842},{"type":"v1","addr":"172.21.15.177:6813","nonce":90842}]},"public_addr":"172.21.15.177:6809/90842","cluster_addr":"172.21.15.177:6811/90842","heartbeat_back_addr":"172.21.15.177:6815/90842","heartbeat_front_addr":"172.21.15.177:6813/90842","state":["exists","up"]},{"osd":5,"uuid":"faefcc9b-4187-42af-9488-8acc48afe27e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":90843},{"type":"v1","addr":"172.21.15.177:6801","nonce":90843}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":90843},{"type":"v1","addr":"172.21.15.177:6803","nonce":90843}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":90843},{"type":"v1","addr":"172.21.15.177:6807","nonce":90843}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":90843},{"type":"v1","addr":"172.21.15.177:6805","nonce":90843}]},"public_addr":"172.21.15.177:6801/90843","cluster_addr":"172.21.15.177:6803/90843","heartbeat_back_addr":"172.21.15.177:6807/90843","heartbeat_front_addr":"172.21.15.177:6805/90843","state":["exists","up"]},{"osd":6,"uuid":"7e8c078d-e95d-4679-803b-6b59d7076a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":90887},{"type":"v1","addr":"172.21.15.177:6825","nonce":90887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":90887},{"type":"v1","addr":"172.21.15.177:6827","nonce":90887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":90887},{"type":"v1","addr":"172.21.15.177:6831","nonce":90887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":90887},{"type":"v1","addr":"172.21.15.177:6829","nonce":90887}]},"public_addr":"172.21.15.177:6825/90887","cluster_addr":"172.21.15.177:6827/90887","heartbeat_back_addr":"172.21.15.177:6831/90887","heartbeat_front_addr":"172.21.15.177:6829/90887","state":["exists","up"]},{"osd":7,"uuid":"1b515708-022a-4aa8-9a84-61b0a99250a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":90844},{"type":"v1","addr":"172.21.15.177:6817","nonce":90844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":90844},{"type":"v1","addr":"172.21.15.177:6819","nonce":90844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":90844},{"type":"v1","addr":"172.21.15.177:6823","nonce":90844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":90844},{"type":"v1","addr":"172.21.15.177:6821","nonce":90844}]},"public_addr":"172.21.15.177:6817/90844","cluster_addr":"172.21.15.177:6819/90844","heartbeat_back_addr":"172.21.15.177:6823/90844","heartbeat_front_addr":"172.21.15.177:6821/90844","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.266353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.203094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.278522+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.285577+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.300922+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.267772+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.248446+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T11:41:45.261772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/1696671501":"2023-10-07T11:43:04.756021+0000","172.21.15.16:6833/1696671501":"2023-10-07T11:43:04.756021+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T11:43:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 msgr2=0x7f3700069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 0x7f3700069590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d00 tx=0x7f3710009120 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 msgr2=0x7f37200aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f371800c8e0 tx=0x7f3718001710 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 shutdown_connections 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3720156ef0 0x7f37200aed40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f37000670e0 0x7f3700069590 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f372006d520 0x7f3720154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 --2- 172.21.15.16:0/26203542 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37200b1c60 0x7f37200ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 >> 172.21.15.16:0/26203542 conn(0x7f3720067e60 msgr2=0x7f3720068230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 shutdown_connections 2023-10-06T11:43:15.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:15.546+0000 7f3724e64700 1 -- 172.21.15.16:0/26203542 wait complete. 2023-10-06T11:43:16.560 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T11:43:16.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.693+0000 7fcd7dc45700 1 Processor -- start 2023-10-06T11:43:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.693+0000 7fcd7dc45700 1 -- start start 2023-10-06T11:43:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd7dc45700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd7dc45700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcd780b3fc0 0x7fcd780b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd7dc45700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd780b2630 con 0x7fcd780b6170 2023-10-06T11:43:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd7dc45700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780b31d0 con 0x7fcd780b3fc0 2023-10-06T11:43:16.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd7dc45700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b4980 con 0x7fcd78159810 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd77fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcd780b3fc0 0x7fcd780b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd777fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd77fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcd780b3fc0 0x7fcd780b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:45810/0 (socket says 172.21.15.16:45810) 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd777fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.16:43926/0 (socket says 172.21.15.16:43926) 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.694+0000 7fcd77fff700 1 -- 172.21.15.16:0/1214450238 learned_addr learned my addr 172.21.15.16:0/1214450238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd77fff700 1 -- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 msgr2=0x7fcd780ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd77fff700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd77fff700 1 -- 172.21.15.16:0/1214450238 >> v1:172.21.15.177:6789/0 conn(0x7fcd780b6170 legacy=0x7fcd780b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd77fff700 1 -- 172.21.15.16:0/1214450238 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780b65e0 con 0x7fcd780b3fc0 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd767fc700 1 -- 172.21.15.16:0/1214450238 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840496360 0 0) 0x7fcd780b2630 con 0x7fcd780b6170 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd777fe700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T11:43:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd77fff700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd780b4440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009f50 tx=0x7fcd70009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19375e6b1f52a918 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd767fc700 1 -- 172.21.15.16:0/1214450238 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000e070 con 0x7fcd780b3fc0 2023-10-06T11:43:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd767fc700 1 -- 172.21.15.16:0/1214450238 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000b9c0 con 0x7fcd780b3fc0 2023-10-06T11:43:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.695+0000 7fcd767fc700 1 -- 172.21.15.16:0/1214450238 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700092e0 con 0x7fcd780b3fc0 2023-10-06T11:43:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 msgr2=0x7fcd780b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd780b4440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009f50 tx=0x7fcd70009840 comp rx=0 tx=0).stop 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1214450238 shutdown_connections 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd780b4440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1214450238 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd78159810 0x7fcd780ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1214450238 >> 172.21.15.16:0/1214450238 conn(0x7fcd7808a1e0 msgr2=0x7fcd78099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1214450238 shutdown_connections 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.696+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1214450238 wait complete. 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.697+0000 7fcd7dc45700 1 Processor -- start 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.697+0000 7fcd7dc45700 1 -- start start 2023-10-06T11:43:16.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd7814fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd780b6170 0x7fcd7809d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 0x7fcd781580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815b730 con 0x7fcd78159810 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815ab20 con 0x7fcd780b3fc0 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd7dc45700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815ae00 con 0x7fcd780b6170 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd77fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 0x7fcd781580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd77fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 0x7fcd781580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43742/0 (socket says 172.21.15.16:43742) 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd77fff700 1 -- 172.21.15.16:0/1906654470 learned_addr learned my addr 172.21.15.16:0/1906654470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd777fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd7814fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd76ffd700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd780b6170 0x7fcd7809d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.698+0000 7fcd77fff700 1 -- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd780b6170 msgr2=0x7fcd7809d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd77fff700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd780b6170 0x7fcd7809d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd77fff700 1 -- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 msgr2=0x7fcd7814fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd77fff700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd7814fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd77fff700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009420 con 0x7fcd78159810 2023-10-06T11:43:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd76ffd700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcd780b6170 0x7fcd7809d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd777fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcd780b3fc0 0x7fcd7814fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd77fff700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 0x7fcd781580b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b6c0 tx=0x7fcd70018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70008c70 con 0x7fcd78159810 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70008dd0 con 0x7fcd78159810 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70009ae0 con 0x7fcd78159810 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7001c640 con 0x7fcd78159810 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.699+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7815b3c0 con 0x7fcd78159810 2023-10-06T11:43:16.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.700+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd78063e70 con 0x7fcd78159810 2023-10-06T11:43:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.700+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fcd70003040 con 0x7fcd78159810 2023-10-06T11:43:16.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd5f7fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcd640670e0 0x7fcd64069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fcd70086040 con 0x7fcd78159810 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd5f7fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7fcd6406d550 0x7fcd6406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd64070030 con 0x7fcd6406d550 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd70088540 con 0x7fcd78159810 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd777fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcd640670e0 0x7fcd64069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd76ffd700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7fcd6406d550 0x7fcd6406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd76ffd700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7fcd6406d550 0x7fcd6406f960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.701+0000 7fcd777fe700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcd640670e0 0x7fcd64069590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd680093f0 tx=0x7fcd68008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.702+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== osd.0 v2:172.21.15.16:6808/90625 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcd64070030 con 0x7fcd6406d550 2023-10-06T11:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 --> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcd7809e290 con 0x7fcd6406d550 2023-10-06T11:43:16.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd5f7fe700 1 -- 172.21.15.16:0/1906654470 <== osd.0 v2:172.21.15.16:6808/90625 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcd7809e290 con 0x7fcd6406d550 2023-10-06T11:43:16.717 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:16.717 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:16.718 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7fcd6406d550 msgr2=0x7fcd6406f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:6808/90625,v1:172.21.15.16:6809/90625] conn(0x7fcd6406d550 0x7fcd6406f960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcd640670e0 msgr2=0x7fcd64069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fcd640670e0 0x7fcd64069590 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcd680093f0 tx=0x7fcd68008040 comp rx=0 tx=0).stop 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 msgr2=0x7fcd781580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd7dc45700 1 --2- 172.21.15.16:0/1906654470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78159810 0x7fcd781580b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000b6c0 tx=0x7fcd70018000 comp rx=0 tx=0).stop 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.716+0000 7fcd77fff700 1 -- 172.21.15.16:0/1906654470 reap_dead start 2023-10-06T11:43:16.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.717+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 shutdown_connections 2023-10-06T11:43:16.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.717+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 >> 172.21.15.16:0/1906654470 conn(0x7fcd7808a1e0 msgr2=0x7fcd7809a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:16.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.717+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 shutdown_connections 2023-10-06T11:43:16.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.717+0000 7fcd7dc45700 1 -- 172.21.15.16:0/1906654470 wait complete. 2023-10-06T11:43:16.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T11:43:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.848+0000 7f072c511700 1 Processor -- start 2023-10-06T11:43:16.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.848+0000 7f072c511700 1 -- start start 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072c511700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f072406d300 0x7f07240cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072c511700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07240cd190 0x7f07240cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072c511700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07240cf060 con 0x7f072415ab50 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072c511700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0724154580 con 0x7f07240cd190 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072c511700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07240cf900 con 0x7f072406d300 2023-10-06T11:43:16.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072aaae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07240cd190 0x7f07240cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072a2ad700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f072415ab50 0x7f072406cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42860/0 (socket says 172.21.15.16:42860) 2023-10-06T11:43:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072a2ad700 1 -- 172.21.15.16:0/84161762 learned_addr learned my addr 172.21.15.16:0/84161762 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f072aaae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07240cd190 0x7f07240cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43752/0 (socket says 172.21.15.16:43752) 2023-10-06T11:43:16.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.849+0000 7f0729aac700 1 --2- 172.21.15.16:0/84161762 >> v2:172.21.15.177:3300/0 conn(0x7f072406d300 0x7f07240cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f072aaae700 1 -- 172.21.15.16:0/84161762 >> v2:172.21.15.177:3300/0 conn(0x7f072406d300 msgr2=0x7f07240cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f072aaae700 1 --2- 172.21.15.16:0/84161762 >> v2:172.21.15.177:3300/0 conn(0x7f072406d300 0x7f07240cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f072aaae700 1 -- 172.21.15.16:0/84161762 >> v1:172.21.15.16:6789/0 conn(0x7f072415ab50 legacy=0x7f072406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f072aaae700 1 -- 172.21.15.16:0/84161762 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240cdec0 con 0x7f07240cd190 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f07292ab700 1 -- 172.21.15.16:0/84161762 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153505622 0 0) 0x7f07240cf060 con 0x7f072415ab50 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f072aaae700 1 --2- 172.21.15.16:0/84161762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f07240cd5d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f072409e550 tx=0x7f071c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638fe7976b1ce504 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f07292ab700 1 -- 172.21.15.16:0/84161762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00b6b0 con 0x7f07240cd190 2023-10-06T11:43:16.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f07292ab700 1 -- 172.21.15.16:0/84161762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c00b810 con 0x7f07240cd190 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.850+0000 7f07292ab700 1 -- 172.21.15.16:0/84161762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c00bbb0 con 0x7f07240cd190 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 -- 172.21.15.16:0/84161762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 msgr2=0x7f07240cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 --2- 172.21.15.16:0/84161762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f07240cd5d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f072409e550 tx=0x7f071c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 -- 172.21.15.16:0/84161762 shutdown_connections 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 --2- 172.21.15.16:0/84161762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f07240cd5d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 --2- 172.21.15.16:0/84161762 >> v2:172.21.15.177:3300/0 conn(0x7f072406d300 0x7f07240cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 -- 172.21.15.16:0/84161762 >> 172.21.15.16:0/84161762 conn(0x7f072408a270 msgr2=0x7f0724099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 -- 172.21.15.16:0/84161762 shutdown_connections 2023-10-06T11:43:16.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.851+0000 7f072c511700 1 -- 172.21.15.16:0/84161762 wait complete. 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.852+0000 7f072c511700 1 Processor -- start 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.852+0000 7f072c511700 1 -- start start 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.852+0000 7f072c511700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 0x7f07240c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.852+0000 7f072c511700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 0x7f07240c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07240c7a90 con 0x7f07240cd190 2023-10-06T11:43:16.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072416ae10 con 0x7f072406d300 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072416bc70 con 0x7f072415ab50 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 -- 172.21.15.16:0/1581532558 learned_addr learned my addr 172.21.15.16:0/1581532558 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072a2ad700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 0x7f07240c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072aaae700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 0x7f07240c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 -- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 msgr2=0x7f07240c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 0x7f07240c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 -- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 msgr2=0x7f07240c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 0x7f07240c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:16.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 -- 172.21.15.16:0/1581532558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c009420 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072a2ad700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 0x7f07240c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072aaae700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 0x7f07240c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f0729aac700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f071800d860 tx=0x7f0718008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718009610 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718009770 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0724154280 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718009b30 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.853+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072416b620 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.854+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072416be20 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.855+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f071801e460 con 0x7f07240cd190 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.855+0000 7f07227fc700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 0x7f0710069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:16.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.855+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0718084d40 con 0x7f07240cd190 2023-10-06T11:43:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.855+0000 7f072a2ad700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 0x7f0710069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.856+0000 7f072a2ad700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 0x7f0710069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07240cbf80 tx=0x7f0714009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:16.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:16.859+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f072416be20 con 0x7f07240cd190 2023-10-06T11:43:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.053+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f072416bde0 con 0x7f07100670e0 2023-10-06T11:43:17.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.053+0000 7f07227fc700 1 -- 172.21.15.16:0/1581532558 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f072416bde0 con 0x7f07100670e0 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 msgr2=0x7f0710069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 0x7f0710069590 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07240cbf80 tx=0x7f0714009040 comp rx=0 tx=0).stop 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 msgr2=0x7f0724153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f071800d860 tx=0x7f0718008040 comp rx=0 tx=0).stop 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 shutdown_connections 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f07100670e0 0x7f0710069590 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f072415ab50 0x7f07240c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f072406d300 0x7f07240c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 --2- 172.21.15.16:0/1581532558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07240cd190 0x7f0724153d40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 >> 172.21.15.16:0/1581532558 conn(0x7f072408a270 msgr2=0x7f07240d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 shutdown_connections 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.055+0000 7f072c511700 1 -- 172.21.15.16:0/1581532558 wait complete. 2023-10-06T11:43:17.058 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T11:43:17.069 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T11:43:17.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.192+0000 7f49bf59e700 1 Processor -- start 2023-10-06T11:43:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.192+0000 7f49bf59e700 1 -- start start 2023-10-06T11:43:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bf59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00b4850 0x7f49c00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bf59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49c00b6030 0x7f49c0163230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bf59e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b33b0 con 0x7f49c00d04e0 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bf59e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c0159a70 con 0x7f49c00b6030 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bf59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b6490 con 0x7f49c00b4850 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49c00b6030 0x7f49c0163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49be59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49c00d04e0 0x7f49c00d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42868/0 (socket says 172.21.15.16:42868) 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49be59c700 1 -- 172.21.15.16:0/700158818 learned_addr learned my addr 172.21.15.16:0/700158818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49c00b6030 0x7f49c0163230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43768/0 (socket says 172.21.15.16:43768) 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bdd9b700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00b4850 0x7f49c00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 -- 172.21.15.16:0/700158818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00b4850 msgr2=0x7f49c00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00b4850 0x7f49c00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 -- 172.21.15.16:0/700158818 >> v1:172.21.15.16:6789/0 conn(0x7f49c00d04e0 legacy=0x7f49c00d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bed9d700 1 -- 172.21.15.16:0/700158818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00cf800 con 0x7f49c00b6030 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.193+0000 7f49bd59a700 1 -- 172.21.15.16:0/700158818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417133925 0 0) 0x7f49c00b33b0 con 0x7f49c00d04e0 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bed9d700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b6030 0x7f49c0163230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098e0 tx=0x7f49b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66a10b3c1385c75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bd59a700 1 -- 172.21.15.16:0/700158818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8009e80 con 0x7f49c00b6030 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bd59a700 1 -- 172.21.15.16:0/700158818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b800b860 con 0x7f49c00b6030 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bd59a700 1 -- 172.21.15.16:0/700158818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800bbe0 con 0x7f49c00b6030 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 -- 172.21.15.16:0/700158818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b6030 msgr2=0x7f49c0163230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b6030 0x7f49c0163230 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098e0 tx=0x7f49b80091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 -- 172.21.15.16:0/700158818 shutdown_connections 2023-10-06T11:43:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b6030 0x7f49c0163230 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 --2- 172.21.15.16:0/700158818 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00b4850 0x7f49c00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 -- 172.21.15.16:0/700158818 >> 172.21.15.16:0/700158818 conn(0x7f49c006cea0 msgr2=0x7f49c006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 -- 172.21.15.16:0/700158818 shutdown_connections 2023-10-06T11:43:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.194+0000 7f49bf59e700 1 -- 172.21.15.16:0/700158818 wait complete. 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.195+0000 7f49bf59e700 1 Processor -- start 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.195+0000 7f49bf59e700 1 -- start start 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 0x7f49c014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c00b6030 0x7f49c00c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00d04e0 0x7f49c00c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c5390 con 0x7f49c00b4850 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c55a0 con 0x7f49c00b6030 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bf59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c5880 con 0x7f49c00d04e0 2023-10-06T11:43:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49be59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 0x7f49c014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49be59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 0x7f49c014a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43778/0 (socket says 172.21.15.16:43778) 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49be59c700 1 -- 172.21.15.16:0/3664746151 learned_addr learned my addr 172.21.15.16:0/3664746151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.196+0000 7f49bdd9b700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c00b6030 0x7f49c00c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 -- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00d04e0 msgr2=0x7f49c00c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f49c00d04e0 0x7f49c00c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 -- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c00b6030 msgr2=0x7f49c00c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49c00b6030 0x7f49c00c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009420 con 0x7f49c00b4850 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49be59c700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 0x7f49c014a020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49b000cd80 tx=0x7f49b0001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000ab10 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000ac70 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00c5b60 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00070f0 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.197+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c014a560 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.198+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49c004c380 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.198+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f49b00074e0 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49b67fc700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f49a00670e0 0x7f49a0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f49b0084c30 con 0x7f49c00b4850 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49b67fc700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f49a006d550 0x7f49a006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49a0070030 con 0x7f49a006d550 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b0087250 con 0x7f49c00b4850 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49bdd9b700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f49a00670e0 0x7f49a0069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49bed9d700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f49a006d550 0x7f49a006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49bdd9b700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f49a00670e0 0x7f49a0069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b2650 tx=0x7f49a8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.199+0000 7f49bed9d700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f49a006d550 0x7f49a006f960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.200+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== osd.1 v2:172.21.15.16:6800/90616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f49a0070030 con 0x7f49a006d550 2023-10-06T11:43:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 --> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f49c00ccdd0 con 0x7f49a006d550 2023-10-06T11:43:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49b67fc700 1 -- 172.21.15.16:0/3664746151 <== osd.1 v2:172.21.15.16:6800/90616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f49c00ccdd0 con 0x7f49a006d550 2023-10-06T11:43:17.215 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:17.215 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:17.216 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:17.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f49a006d550 msgr2=0x7f49a006f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:6800/90616,v1:172.21.15.16:6801/90616] conn(0x7f49a006d550 0x7f49a006f960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f49a00670e0 msgr2=0x7f49a0069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f49a00670e0 0x7f49a0069590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b2650 tx=0x7f49a8008040 comp rx=0 tx=0).stop 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 msgr2=0x7f49c014a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 --2- 172.21.15.16:0/3664746151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49c00b4850 0x7f49c014a020 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f49b000cd80 tx=0x7f49b0001ae0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bed9d700 1 -- 172.21.15.16:0/3664746151 reap_dead start 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 shutdown_connections 2023-10-06T11:43:17.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.214+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 >> 172.21.15.16:0/3664746151 conn(0x7f49c006cea0 msgr2=0x7f49c00cebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.215+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 shutdown_connections 2023-10-06T11:43:17.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.215+0000 7f49bf59e700 1 -- 172.21.15.16:0/3664746151 wait complete. 2023-10-06T11:43:17.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T11:43:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.346+0000 7f8771c49700 1 Processor -- start 2023-10-06T11:43:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.346+0000 7f8771c49700 1 -- start start 2023-10-06T11:43:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f8771c49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c067ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f8771c49700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0b4100 con 0x7f876c157870 2023-10-06T11:43:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f8771c49700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c068800 con 0x7f876c068410 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f8771c49700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b34f0 con 0x7f876c1574a0 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f876c157870 0x7f876c0b2700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42882/0 (socket says 172.21.15.16:42882) 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876affd700 1 -- 172.21.15.16:0/1200412258 learned_addr learned my addr 172.21.15.16:0/1200412258 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876b7fe700 1 --2- 172.21.15.16:0/1200412258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c067ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900635954 0 0) 0x7f876c0b4100 con 0x7f876c157870 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8754003570 con 0x7f876c157870 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683424599 0 0) 0x7f876c068800 con 0x7f876c068410 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f876c0b4100 con 0x7f876c068410 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876b7fe700 1 -- 172.21.15.16:0/1200412258 >> v1:172.21.15.177:6789/0 conn(0x7f876c068410 legacy=0x7f876c0ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876b7fe700 1 -- 172.21.15.16:0/1200412258 >> v1:172.21.15.16:6789/0 conn(0x7f876c157870 legacy=0x7f876c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876b7fe700 1 -- 172.21.15.16:0/1200412258 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0b37d0 con 0x7f876c1574a0 2023-10-06T11:43:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.347+0000 7f876b7fe700 1 --2- 172.21.15.16:0/1200412258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c067ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8760009910 tx=0x7f87600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0003040d8c2ac0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760011630 con 0x7f876c1574a0 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8760011790 con 0x7f876c1574a0 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f876a7fc700 1 -- 172.21.15.16:0/1200412258 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760011b30 con 0x7f876c1574a0 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 -- 172.21.15.16:0/1200412258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 msgr2=0x7f876c067ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 --2- 172.21.15.16:0/1200412258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c067ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8760009910 tx=0x7f87600091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 -- 172.21.15.16:0/1200412258 shutdown_connections 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 --2- 172.21.15.16:0/1200412258 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c067ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 -- 172.21.15.16:0/1200412258 >> 172.21.15.16:0/1200412258 conn(0x7f876c06c410 msgr2=0x7f876c06c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 -- 172.21.15.16:0/1200412258 shutdown_connections 2023-10-06T11:43:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.348+0000 7f8771c49700 1 -- 172.21.15.16:0/1200412258 wait complete. 2023-10-06T11:43:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.349+0000 7f8771c49700 1 Processor -- start 2023-10-06T11:43:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 -- start start 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f876c157870 0x7f876c0af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b0250 con 0x7f876c068410 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0c7e90 con 0x7f876c157870 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f8771c49700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c06d5c0 con 0x7f876c1574a0 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f876b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f876b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43784/0 (socket says 172.21.15.16:43784) 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f876b7fe700 1 -- 172.21.15.16:0/322311893 learned_addr learned my addr 172.21.15.16:0/322311893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.350+0000 7f876bfff700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f876c157870 0x7f876c0af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876affd700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 -- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 msgr2=0x7f876c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 -- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f876c157870 msgr2=0x7f876c0af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f876c157870 0x7f876c0af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 -- 172.21.15.16:0/322311893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8760009420 con 0x7f876c068410 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876affd700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f876b7fe700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8760009f00 tx=0x7f8760017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876001f070 con 0x7f876c068410 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8760000bc0 con 0x7f876c068410 2023-10-06T11:43:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c06d8a0 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760025420 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.351+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c06de60 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.352+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c063e70 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.352+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f8760023070 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.353+0000 7f87537fe700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 0x7f8758069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.353+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f8760085b40 con 0x7f876c068410 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.353+0000 7f876affd700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 0x7f8758069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.353+0000 7f876affd700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 0x7f8758069590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f875c007fd0 tx=0x7f875c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.357+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8760052cd0 con 0x7f876c068410 2023-10-06T11:43:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.554+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f876c06cbb0 con 0x7f87580670e0 2023-10-06T11:43:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.555+0000 7f87537fe700 1 -- 172.21.15.16:0/322311893 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f876c06cbb0 con 0x7f87580670e0 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 msgr2=0x7f8758069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 0x7f8758069590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f875c007fd0 tx=0x7f875c009040 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 msgr2=0x7f876c0c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8760009f00 tx=0x7f8760017000 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 shutdown_connections 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f876c068410 0x7f876c0c7790 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f87580670e0 0x7f8758069590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f876c157870 0x7f876c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 --2- 172.21.15.16:0/322311893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f876c1574a0 0x7f876c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 >> 172.21.15.16:0/322311893 conn(0x7f876c06c410 msgr2=0x7f876c156540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 shutdown_connections 2023-10-06T11:43:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.557+0000 7f8771c49700 1 -- 172.21.15.16:0/322311893 wait complete. 2023-10-06T11:43:17.560 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T11:43:17.570 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T11:43:17.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.693+0000 7f60ad5b3700 1 Processor -- start 2023-10-06T11:43:17.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.693+0000 7f60ad5b3700 1 -- start start 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.693+0000 7f60ad5b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60a80b2e10 0x7f60a80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60ad5b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60ad5b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a80cca70 con 0x7f60a80b2a40 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60ad5b3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a80cf950 con 0x7f60a80b2e10 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60ad5b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80cdbb0 con 0x7f60a80cd730 2023-10-06T11:43:17.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f60a80b2a40 0x7f60a80cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42898/0 (socket says 172.21.15.16:42898) 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a6ffd700 1 -- 172.21.15.16:0/3856856122 learned_addr learned my addr 172.21.15.16:0/3856856122 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 --2- 172.21.15.16:0/3856856122 >> v2:172.21.15.16:3300/0 conn(0x7f60a80b2e10 0x7f60a80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a5ffb700 1 -- 172.21.15.16:0/3856856122 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179863572 0 0) 0x7f60a80cca70 con 0x7f60a80b2a40 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a5ffb700 1 -- 172.21.15.16:0/3856856122 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6094003570 con 0x7f60a80b2a40 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a77fe700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 -- 172.21.15.16:0/3856856122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 msgr2=0x7f60a80c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a80c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 -- 172.21.15.16:0/3856856122 >> v1:172.21.15.16:6789/0 conn(0x7f60a80b2a40 legacy=0x7f60a80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 -- 172.21.15.16:0/3856856122 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a80cbe60 con 0x7f60a80b2e10 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a77fe700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a80c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:17.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a67fc700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80cd1f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6090009910 tx=0x7f60900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1bb98119f24a0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a5ffb700 1 -- 172.21.15.16:0/3856856122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000b640 con 0x7f60a80b2e10 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a5ffb700 1 -- 172.21.15.16:0/3856856122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000b7a0 con 0x7f60a80b2e10 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.694+0000 7f60a5ffb700 1 -- 172.21.15.16:0/3856856122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000bb60 con 0x7f60a80b2e10 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 -- 172.21.15.16:0/3856856122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 msgr2=0x7f60a80cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80cd1f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6090009910 tx=0x7f60900091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 -- 172.21.15.16:0/3856856122 shutdown_connections 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a80c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/3856856122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80cd1f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 -- 172.21.15.16:0/3856856122 >> 172.21.15.16:0/3856856122 conn(0x7f60a808a230 msgr2=0x7f60a8099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 -- 172.21.15.16:0/3856856122 shutdown_connections 2023-10-06T11:43:17.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.695+0000 7f60ad5b3700 1 -- 172.21.15.16:0/3856856122 wait complete. 2023-10-06T11:43:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.696+0000 7f60ad5b3700 1 Processor -- start 2023-10-06T11:43:17.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.696+0000 7f60ad5b3700 1 -- start start 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a80b2a40 0x7f60a809c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a8158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80c7de0 con 0x7f60a80b2e10 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80c8c00 con 0x7f60a80b2a40 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60ad5b3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80c7ff0 con 0x7f60a80cd730 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43806/0 (socket says 172.21.15.16:43806) 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 -- 172.21.15.16:0/932993869 learned_addr learned my addr 172.21.15.16:0/932993869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:17.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a77fe700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a8158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a6ffd700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a80b2a40 0x7f60a809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 -- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 msgr2=0x7f60a8158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60a80cd730 0x7f60a8158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 -- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a80b2a40 msgr2=0x7f60a809c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60a80b2a40 0x7f60a809c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6090009420 con 0x7f60a80b2e10 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.697+0000 7f60a67fc700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80c78a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f60a8088e30 tx=0x7f6090017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609001f070 con 0x7f60a80b2e10 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6090000c90 con 0x7f60a80b2e10 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a80c82d0 con 0x7f60a80b2e10 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000bb60 con 0x7f60a80b2e10 2023-10-06T11:43:17.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a80c8890 con 0x7f60a80b2e10 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.698+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60a8063e70 con 0x7f60a80b2e10 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.699+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f6090023070 con 0x7f60a80b2e10 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.699+0000 7f609effd700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f60880670e0 0x7f6088069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.699+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f6090085b00 con 0x7f60a80b2e10 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f60a6ffd700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f60880670e0 0x7f6088069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f609effd700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f608806d550 0x7f608806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6088070030 con 0x7f608806d550 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f609008a220 con 0x7f60a80b2e10 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f60a77fe700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f608806d550 0x7f608806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f60a6ffd700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f60880670e0 0x7f6088069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6098001360 tx=0x7f6098009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.700+0000 7f60a77fe700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f608806d550 0x7f608806f960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.701+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== osd.2 v2:172.21.15.16:6819/90626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6088070030 con 0x7f608806d550 2023-10-06T11:43:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.714+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 --> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60a815e310 con 0x7f608806d550 2023-10-06T11:43:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f609effd700 1 -- 172.21.15.16:0/932993869 <== osd.2 v2:172.21.15.16:6819/90626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f60a815e310 con 0x7f608806d550 2023-10-06T11:43:17.716 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:17.716 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:17.716 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f608806d550 msgr2=0x7f608806f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:6819/90626,v1:172.21.15.16:6821/90626] conn(0x7f608806d550 0x7f608806f960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f60880670e0 msgr2=0x7f6088069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f60880670e0 0x7f6088069590 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6098001360 tx=0x7f6098009040 comp rx=0 tx=0).stop 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 msgr2=0x7f60a80c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 --2- 172.21.15.16:0/932993869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60a80b2e10 0x7f60a80c78a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f60a8088e30 tx=0x7f6090017000 comp rx=0 tx=0).stop 2023-10-06T11:43:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60a77fe700 1 -- 172.21.15.16:0/932993869 reap_dead start 2023-10-06T11:43:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 shutdown_connections 2023-10-06T11:43:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 >> 172.21.15.16:0/932993869 conn(0x7f60a808a230 msgr2=0x7f60a806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 shutdown_connections 2023-10-06T11:43:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.715+0000 7f60ad5b3700 1 -- 172.21.15.16:0/932993869 wait complete. 2023-10-06T11:43:17.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T11:43:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.846+0000 7fd05c18d700 1 Processor -- start 2023-10-06T11:43:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.846+0000 7fd05c18d700 1 -- start start 2023-10-06T11:43:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd05c18d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd054159800 0x7fd0540b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd05c18d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540b0d50 con 0x7fd0540b40b0 2023-10-06T11:43:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd05c18d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540b0140 con 0x7fd0540b6360 2023-10-06T11:43:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd05c18d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540b0420 con 0x7fd054159800 2023-10-06T11:43:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd054159800 0x7fd0540b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd054159800 0x7fd0540b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43816/0 (socket says 172.21.15.16:43816) 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 -- 172.21.15.16:0/2828786719 learned_addr learned my addr 172.21.15.16:0/2828786719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 -- 172.21.15.16:0/2828786719 >> v1:172.21.15.177:6789/0 conn(0x7fd0540b6360 legacy=0x7fd0540b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 -- 172.21.15.16:0/2828786719 >> v1:172.21.15.16:6789/0 conn(0x7fd0540b40b0 legacy=0x7fd0540cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 -- 172.21.15.16:0/2828786719 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540b0700 con 0x7fd054159800 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd059f29700 1 --2- 172.21.15.16:0/2828786719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd054159800 0x7fd0540b2a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd044009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916c448a7061103f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.847+0000 7fd058f27700 1 -- 172.21.15.16:0/2828786719 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04400e070 con 0x7fd054159800 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd058f27700 1 -- 172.21.15.16:0/2828786719 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04400b980 con 0x7fd054159800 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd058f27700 1 -- 172.21.15.16:0/2828786719 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0440092e0 con 0x7fd054159800 2023-10-06T11:43:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 -- 172.21.15.16:0/2828786719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd054159800 msgr2=0x7fd0540b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 --2- 172.21.15.16:0/2828786719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd054159800 0x7fd0540b2a70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd044009840 comp rx=0 tx=0).stop 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 -- 172.21.15.16:0/2828786719 shutdown_connections 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 --2- 172.21.15.16:0/2828786719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd054159800 0x7fd0540b2a70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 -- 172.21.15.16:0/2828786719 >> 172.21.15.16:0/2828786719 conn(0x7fd05408a230 msgr2=0x7fd054099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 -- 172.21.15.16:0/2828786719 shutdown_connections 2023-10-06T11:43:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.848+0000 7fd05c18d700 1 -- 172.21.15.16:0/2828786719 wait complete. 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.849+0000 7fd05c18d700 1 Processor -- start 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.849+0000 7fd05c18d700 1 -- start start 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 0x7fd0540aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 0x7fd05409bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540ad5d0 con 0x7fd0540b40b0 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540ad140 con 0x7fd054159800 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05c18d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0540ac530 con 0x7fd0540b6360 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43826/0 (socket says 172.21.15.16:43826) 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 -- 172.21.15.16:0/1974326421 learned_addr learned my addr 172.21.15.16:0/1974326421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 -- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 msgr2=0x7fd0540aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059728700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 0x7fd0540aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05a72a700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 0x7fd05409bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 0x7fd0540aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 -- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 msgr2=0x7fd05409bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 0x7fd05409bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059f29700 1 -- 172.21.15.16:0/1974326421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd044009ae0 con 0x7fd0540b40b0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd059728700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 0x7fd0540aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.850+0000 7fd05a72a700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 0x7fd05409bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd059f29700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd04400b6c0 tx=0x7fd044016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0440127c0 con 0x7fd0540b40b0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd044012920 con 0x7fd0540b40b0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044012ce0 con 0x7fd0540b40b0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0540ac810 con 0x7fd0540b40b0 2023-10-06T11:43:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.851+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0540acdd0 con 0x7fd0540b40b0 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.852+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0540ad420 con 0x7fd0540b40b0 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.852+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd04401e690 con 0x7fd0540b40b0 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.853+0000 7fd051ffb700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 0x7fd040069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.853+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd044086230 con 0x7fd0540b40b0 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.853+0000 7fd059728700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 0x7fd040069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.853+0000 7fd059728700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 0x7fd040069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0540ab080 tx=0x7fd048009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:17.857+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0440532b0 con 0x7fd0540b40b0 2023-10-06T11:43:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.051+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0540ad420 con 0x7fd0400670e0 2023-10-06T11:43:18.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.051+0000 7fd051ffb700 1 -- 172.21.15.16:0/1974326421 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd0540ad420 con 0x7fd0400670e0 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 msgr2=0x7fd040069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 0x7fd040069590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0540ab080 tx=0x7fd048009040 comp rx=0 tx=0).stop 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 msgr2=0x7fd054156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd04400b6c0 tx=0x7fd044016000 comp rx=0 tx=0).stop 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 shutdown_connections 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540b40b0 0x7fd054156e60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd0400670e0 0x7fd040069590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd054159800 0x7fd05409bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 --2- 172.21.15.16:0/1974326421 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd0540b6360 0x7fd0540aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 >> 172.21.15.16:0/1974326421 conn(0x7fd05408a230 msgr2=0x7fd054158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 shutdown_connections 2023-10-06T11:43:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.053+0000 7fd05c18d700 1 -- 172.21.15.16:0/1974326421 wait complete. 2023-10-06T11:43:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T11:43:18.066 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T11:43:18.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.189+0000 7f2ffcde9700 1 Processor -- start 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.189+0000 7f2ffcde9700 1 -- start start 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ffcde9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 0x7f2ff80ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ffcde9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b2630 con 0x7f2ff80b6170 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ffcde9700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b65e0 con 0x7f2ff80b3fc0 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ffcde9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80b31d0 con 0x7f2ff8159810 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff5d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ff80b6170 0x7f2ff80b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42918/0 (socket says 172.21.15.16:42918) 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff5d9b700 1 -- 172.21.15.16:0/3910755792 learned_addr learned my addr 172.21.15.16:0/3910755792 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:18.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff659c700 1 --2- 172.21.15.16:0/3910755792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 0x7f2ff80ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021797644 0 0) 0x7f2ff80b2630 con 0x7f2ff80b6170 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe0003570 con 0x7f2ff80b6170 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1741235085 0 0) 0x7f2fe0003570 con 0x7f2ff80b6170 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff659c700 1 -- 172.21.15.16:0/3910755792 >> v1:172.21.15.177:6789/0 conn(0x7f2ff80b3fc0 legacy=0x7f2ff8159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff659c700 1 -- 172.21.15.16:0/3910755792 >> v1:172.21.15.16:6789/0 conn(0x7f2ff80b6170 legacy=0x7f2ff80b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.190+0000 7f2ff659c700 1 -- 172.21.15.16:0/3910755792 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80b1a80 con 0x7f2ff8159810 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ff659c700 1 --2- 172.21.15.16:0/3910755792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 0x7f2ff80ab040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009910 tx=0x7f2fe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd107c39b658b7f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec001470 con 0x7f2ff8159810 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8011420 con 0x7f2ff8159810 2023-10-06T11:43:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ff559a700 1 -- 172.21.15.16:0/3910755792 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80117e0 con 0x7f2ff8159810 2023-10-06T11:43:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3910755792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 msgr2=0x7f2ff80ab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ffcde9700 1 --2- 172.21.15.16:0/3910755792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 0x7f2ff80ab040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009910 tx=0x7f2fe80091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3910755792 shutdown_connections 2023-10-06T11:43:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ffcde9700 1 --2- 172.21.15.16:0/3910755792 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff8159810 0x7f2ff80ab040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.191+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3910755792 >> 172.21.15.16:0/3910755792 conn(0x7f2ff808a1e0 msgr2=0x7f2ff8099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.192+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3910755792 shutdown_connections 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.192+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3910755792 wait complete. 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 Processor -- start 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 -- start start 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff80b3fc0 0x7f2ff8158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 0x7f2ff80ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff8159810 0x7f2ff80ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8169c30 con 0x7f2ff80b6170 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80ae050 con 0x7f2ff8159810 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.193+0000 7f2ffcde9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80ae330 con 0x7f2ff80b3fc0 2023-10-06T11:43:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 0x7f2ff80ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 0x7f2ff80ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43838/0 (socket says 172.21.15.16:43838) 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 -- 172.21.15.16:0/3295389325 learned_addr learned my addr 172.21.15.16:0/3295389325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff659c700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff80b3fc0 0x7f2ff8158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff6d9d700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff8159810 0x7f2ff80ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 -- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff80b3fc0 msgr2=0x7f2ff8158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2ff80b3fc0 0x7f2ff8158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 -- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff8159810 msgr2=0x7f2ff80ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2ff8159810 0x7f2ff80ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe8009420 con 0x7f2ff80b6170 2023-10-06T11:43:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ff5d9b700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 0x7f2ff80ad950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00cb90 tx=0x7f2fec001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec001d60 con 0x7f2ff80b6170 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec007360 con 0x7f2ff80b6170 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff80ae610 con 0x7f2ff80b6170 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec007700 con 0x7f2ff80b6170 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.194+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff815c370 con 0x7f2ff80b6170 2023-10-06T11:43:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.195+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ff8063e70 con 0x7f2ff80b6170 2023-10-06T11:43:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f2fec004a40 con 0x7f2ff80b6170 2023-10-06T11:43:18.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2fd80670e0 0x7f2fd8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2fec083e60 con 0x7f2ff80b6170 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f2fd806d550 0x7f2fd806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fd8070030 con 0x7f2fd806d550 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fec086480 con 0x7f2ff80b6170 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.196+0000 7f2ff6d9d700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f2fd806d550 0x7f2fd806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.197+0000 7f2ff6d9d700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f2fd806d550 0x7f2fd806f960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.197+0000 7f2ff659c700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2fd80670e0 0x7f2fd8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.197+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== osd.3 v2:172.21.15.16:6816/90633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2fd8070030 con 0x7f2fd806d550 2023-10-06T11:43:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.198+0000 7f2ff659c700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2fd80670e0 0x7f2fd8069590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009860 tx=0x7f2fe8016000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.211+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 --> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ff80b1d00 con 0x7f2fd806d550 2023-10-06T11:43:18.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.211+0000 7f2fe67fc700 1 -- 172.21.15.16:0/3295389325 <== osd.3 v2:172.21.15.16:6816/90633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2ff80b1d00 con 0x7f2fd806d550 2023-10-06T11:43:18.212 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:18.213 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.211+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f2fd806d550 msgr2=0x7f2fd806f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.211+0000 7f2ffcde9700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:6816/90633,v1:172.21.15.16:6817/90633] conn(0x7f2fd806d550 0x7f2fd806f960 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2fd80670e0 msgr2=0x7f2fd8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f2fd80670e0 0x7f2fd8069590 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009860 tx=0x7f2fe8016000 comp rx=0 tx=0).stop 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 msgr2=0x7f2ff80ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 --2- 172.21.15.16:0/3295389325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ff80b6170 0x7f2ff80ad950 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00cb90 tx=0x7f2fec001ad0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ff6d9d700 1 -- 172.21.15.16:0/3295389325 reap_dead start 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 shutdown_connections 2023-10-06T11:43:18.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 >> 172.21.15.16:0/3295389325 conn(0x7f2ff808a1e0 msgr2=0x7f2ff809a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 shutdown_connections 2023-10-06T11:43:18.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.212+0000 7f2ffcde9700 1 -- 172.21.15.16:0/3295389325 wait complete. 2023-10-06T11:43:18.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T11:43:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 Processor -- start 2023-10-06T11:43:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 -- start start 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ee8159800 0x7f4ee80b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 0x7f4ee80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee80b0d50 con 0x7f4ee80b1720 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80b0140 con 0x7f4ee8159800 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4eed5b9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80b0420 con 0x7f4ee80b6360 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.343+0000 7f4ee6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ee8159800 0x7f4ee80b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ee80b1720 0x7f4ee80b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59024/0 (socket says 172.21.15.16:59024) 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee77fe700 1 -- 172.21.15.16:0/49054353 learned_addr learned my addr 172.21.15.16:0/49054353 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee67fc700 1 --2- 172.21.15.16:0/49054353 >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 0x7f4ee80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee6ffd700 1 -- 172.21.15.16:0/49054353 >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 msgr2=0x7f4ee80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee6ffd700 1 --2- 172.21.15.16:0/49054353 >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 0x7f4ee80b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee6ffd700 1 -- 172.21.15.16:0/49054353 >> v1:172.21.15.16:6789/0 conn(0x7f4ee80b1720 legacy=0x7f4ee80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee6ffd700 1 -- 172.21.15.16:0/49054353 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80b0700 con 0x7f4ee8159800 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee67fc700 1 --2- 172.21.15.16:0/49054353 >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 0x7f4ee80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee6ffd700 1 --2- 172.21.15.16:0/49054353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee80b2a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80b6830 tx=0x7f4ed8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412cdbb3cb65a5d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee5ffb700 1 -- 172.21.15.16:0/49054353 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed800e070 con 0x7f4ee8159800 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee5ffb700 1 -- 172.21.15.16:0/49054353 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed800bc30 con 0x7f4ee8159800 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.344+0000 7f4ee5ffb700 1 -- 172.21.15.16:0/49054353 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8012430 con 0x7f4ee8159800 2023-10-06T11:43:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 -- 172.21.15.16:0/49054353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 msgr2=0x7f4ee80b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/49054353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee80b2a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80b6830 tx=0x7f4ed8009780 comp rx=0 tx=0).stop 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 -- 172.21.15.16:0/49054353 shutdown_connections 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/49054353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee80b2a70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/49054353 >> v2:172.21.15.177:3300/0 conn(0x7f4ee80b6360 0x7f4ee80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 -- 172.21.15.16:0/49054353 >> 172.21.15.16:0/49054353 conn(0x7f4ee808a230 msgr2=0x7f4ee8099b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 -- 172.21.15.16:0/49054353 shutdown_connections 2023-10-06T11:43:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.345+0000 7f4eed5b9700 1 -- 172.21.15.16:0/49054353 wait complete. 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.346+0000 7f4eed5b9700 1 Processor -- start 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.346+0000 7f4eed5b9700 1 -- start start 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ee80b1720 0x7f4ee8151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 0x7f4ee809cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806dd00 con 0x7f4ee8159800 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806e880 con 0x7f4ee80b6360 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806ed10 con 0x7f4ee80b1720 2023-10-06T11:43:18.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38288/0 (socket says 172.21.15.16:38288) 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 -- 172.21.15.16:0/3634451266 learned_addr learned my addr 172.21.15.16:0/3634451266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee67fc700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 0x7f4ee809cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee6ffd700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ee80b1720 0x7f4ee8151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 -- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ee80b1720 msgr2=0x7f4ee8151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ee80b1720 0x7f4ee8151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 -- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 msgr2=0x7f4ee809cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 0x7f4ee809cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 -- 172.21.15.16:0/3634451266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8009a20 con 0x7f4ee8159800 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee67fc700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 0x7f4ee809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4ee77fe700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c560 tx=0x7f4ee0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0005b30 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0005c90 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee806df50 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00072e0 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.347+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee806e510 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.348+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee806ea30 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.349+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f4ee0007690 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.349+0000 7f4edeffd700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 0x7f4ec8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.349+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f4ee00839e0 con 0x7f4ee8159800 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.349+0000 7f4ee6ffd700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 0x7f4ec8069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.350+0000 7f4ee6ffd700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 0x7f4ec8069590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80b6830 tx=0x7f4ed8019000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.353+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ee004b5a0 con 0x7f4ee8159800 2023-10-06T11:43:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.546+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ee8063e70 con 0x7f4ec80670e0 2023-10-06T11:43:18.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.547+0000 7f4edeffd700 1 -- 172.21.15.16:0/3634451266 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ee8063e70 con 0x7f4ec80670e0 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 msgr2=0x7f4ec8069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 0x7f4ec8069590 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80b6830 tx=0x7f4ed8019000 comp rx=0 tx=0).stop 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 msgr2=0x7f4ee809d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c560 tx=0x7f4ee0007f60 comp rx=0 tx=0).stop 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 shutdown_connections 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ee8159800 0x7f4ee809d280 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f4ec80670e0 0x7f4ec8069590 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ee80b1720 0x7f4ee8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 --2- 172.21.15.16:0/3634451266 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ee80b6360 0x7f4ee809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 >> 172.21.15.16:0/3634451266 conn(0x7f4ee808a230 msgr2=0x7f4ee8154140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 shutdown_connections 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.549+0000 7f4eed5b9700 1 -- 172.21.15.16:0/3634451266 wait complete. 2023-10-06T11:43:18.551 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T11:43:18.562 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T11:43:18.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.685+0000 7fd8f1b3b700 1 Processor -- start 2023-10-06T11:43:18.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.685+0000 7fd8f1b3b700 1 -- start start 2023-10-06T11:43:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f1b3b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd8ec156fa0 0x7fd8ec15a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f1b3b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 0x7fd8ec0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f1b3b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ec0cebc0 con 0x7fd8ec157370 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f1b3b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0cee60 con 0x7fd8ec156fa0 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f1b3b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0cf140 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8ebfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8ec157370 0x7fd8ec0d03a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59028/0 (socket says 172.21.15.16:59028) 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8ebfff700 1 -- 172.21.15.16:0/1512022715 learned_addr learned my addr 172.21.15.16:0/1512022715 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f133a700 1 --2- 172.21.15.16:0/1512022715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 0x7fd8ec0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8f0b39700 1 --2- 172.21.15.16:0/1512022715 >> v2:172.21.15.177:3300/0 conn(0x7fd8ec156fa0 0x7fd8ec15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181152751 0 0) 0x7fd8ec0cebc0 con 0x7fd8ec157370 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.686+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d8003570 con 0x7fd8ec157370 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f133a700 1 -- 172.21.15.16:0/1512022715 >> v2:172.21.15.177:3300/0 conn(0x7fd8ec156fa0 msgr2=0x7fd8ec15a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:18.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f133a700 1 --2- 172.21.15.16:0/1512022715 >> v2:172.21.15.177:3300/0 conn(0x7fd8ec156fa0 0x7fd8ec15a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f133a700 1 -- 172.21.15.16:0/1512022715 >> v1:172.21.15.16:6789/0 conn(0x7fd8ec157370 legacy=0x7fd8ec0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3030035402 0 0) 0x7fd8d8003570 con 0x7fd8ec157370 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f133a700 1 -- 172.21.15.16:0/1512022715 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0cf420 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f133a700 1 --2- 172.21.15.16:0/1512022715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 0x7fd8ec0cfc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40098e0 tx=0x7fd8d40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783d92a6c3d99cdb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc001470 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4011420 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8eb7fe700 1 -- 172.21.15.16:0/1512022715 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40117c0 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/1512022715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 msgr2=0x7fd8ec0cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/1512022715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 0x7fd8ec0cfc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd8d40098e0 tx=0x7fd8d40091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/1512022715 shutdown_connections 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/1512022715 >> v2:172.21.15.177:3300/0 conn(0x7fd8ec156fa0 0x7fd8ec15a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.687+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/1512022715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec0d0ab0 0x7fd8ec0cfc20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.688+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/1512022715 >> 172.21.15.16:0/1512022715 conn(0x7fd8ec06cea0 msgr2=0x7fd8ec06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.688+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/1512022715 shutdown_connections 2023-10-06T11:43:18.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.688+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/1512022715 wait complete. 2023-10-06T11:43:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.689+0000 7fd8f1b3b700 1 Processor -- start 2023-10-06T11:43:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.689+0000 7fd8f1b3b700 1 -- start start 2023-10-06T11:43:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 0x7fd8ec16cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8ec156fa0 0x7fd8ec15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec157370 0x7fd8ec0b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec16de30 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec16d590 con 0x7fd8ec156fa0 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec16d870 con 0x7fd8ec157370 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 0x7fd8ec16cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 0x7fd8ec16cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38292/0 (socket says 172.21.15.16:38292) 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 -- 172.21.15.16:0/333524388 learned_addr learned my addr 172.21.15.16:0/333524388 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8ebfff700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8ec156fa0 0x7fd8ec15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f133a700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec157370 0x7fd8ec0b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 -- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec157370 msgr2=0x7fd8ec0b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd8ec157370 0x7fd8ec0b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 -- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8ec156fa0 msgr2=0x7fd8ec15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd8ec156fa0 0x7fd8ec15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4009420 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f0b39700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 0x7fd8ec16cfb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009f40 tx=0x7fd8e400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400cb70 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400f040 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec16db50 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.690+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4014480 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.691+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec0b1fd0 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.691+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd8ec0b21f0 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd8e40147f0 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8cc0670e0 0x7fd8cc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd8e4084f50 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd8cc06d550 0x7fd8cc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8cc070030 con 0x7fd8cc06d550 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8e40874b0 con 0x7fd8ec0d0ab0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.692+0000 7fd8ebfff700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8cc0670e0 0x7fd8cc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.693+0000 7fd8f133a700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd8cc06d550 0x7fd8cc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.693+0000 7fd8ebfff700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8cc0670e0 0x7fd8cc069590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec15dd00 tx=0x7fd8dc001a10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.693+0000 7fd8f133a700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd8cc06d550 0x7fd8cc06f960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.694+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== osd.4 v2:172.21.15.177:6808/90842 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd8cc070030 con 0x7fd8cc06d550 2023-10-06T11:43:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 --> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8ec04c380 con 0x7fd8cc06d550 2023-10-06T11:43:18.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8e8ff9700 1 -- 172.21.15.16:0/333524388 <== osd.4 v2:172.21.15.177:6808/90842 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd8ec04c380 con 0x7fd8cc06d550 2023-10-06T11:43:18.710 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:18.710 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:18.710 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd8cc06d550 msgr2=0x7fd8cc06f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6808/90842,v1:172.21.15.177:6809/90842] conn(0x7fd8cc06d550 0x7fd8cc06f960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8cc0670e0 msgr2=0x7fd8cc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd8cc0670e0 0x7fd8cc069590 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec15dd00 tx=0x7fd8dc001a10 comp rx=0 tx=0).stop 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 msgr2=0x7fd8ec16cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 --2- 172.21.15.16:0/333524388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ec0d0ab0 0x7fd8ec16cfb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009f40 tx=0x7fd8e400b040 comp rx=0 tx=0).stop 2023-10-06T11:43:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f133a700 1 -- 172.21.15.16:0/333524388 reap_dead start 2023-10-06T11:43:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 shutdown_connections 2023-10-06T11:43:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 >> 172.21.15.16:0/333524388 conn(0x7fd8ec06cea0 msgr2=0x7fd8ec0d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 shutdown_connections 2023-10-06T11:43:18.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.708+0000 7fd8f1b3b700 1 -- 172.21.15.16:0/333524388 wait complete. 2023-10-06T11:43:18.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T11:43:18.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.839+0000 7efd6b4f0700 1 Processor -- start 2023-10-06T11:43:18.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 -- start start 2023-10-06T11:43:18.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd6406d300 0x7efd640cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd640cd190 0x7efd640cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd640cf060 con 0x7efd6415ab50 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd64154580 con 0x7efd6406d300 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.840+0000 7efd6b4f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cf900 con 0x7efd640cd190 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd6406d300 0x7efd640cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd69a8d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd640cd190 0x7efd640cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd6406d300 0x7efd640cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.16:55110/0 (socket says 172.21.15.16:55110) 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd6928c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7efd6415ab50 0x7efd6406cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.16:50416/0 (socket says 172.21.15.16:50416) 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 -- 172.21.15.16:0/2937761726 learned_addr learned my addr 172.21.15.16:0/2937761726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 -- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd640cd190 msgr2=0x7efd640cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 --2- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd640cd190 0x7efd640cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 -- 172.21.15.16:0/2937761726 >> v1:172.21.15.177:6789/0 conn(0x7efd6415ab50 legacy=0x7efd6406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 -- 172.21.15.16:0/2937761726 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640cdec0 con 0x7efd6406d300 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd63fff700 1 -- 172.21.15.16:0/2937761726 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264626882 0 0) 0x7efd640cf060 con 0x7efd6415ab50 2023-10-06T11:43:18.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd68a8b700 1 --2- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6406d300 0x7efd640cfd90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd580099b0 tx=0x7efd58009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88eee21152be63ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd63fff700 1 -- 172.21.15.16:0/2937761726 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58009ea0 con 0x7efd6406d300 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd63fff700 1 -- 172.21.15.16:0/2937761726 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5800b860 con 0x7efd6406d300 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.841+0000 7efd63fff700 1 -- 172.21.15.16:0/2937761726 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800bc00 con 0x7efd6406d300 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6406d300 msgr2=0x7efd640cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6406d300 0x7efd640cfd90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd580099b0 tx=0x7efd58009250 comp rx=0 tx=0).stop 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2937761726 shutdown_connections 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd640cd190 0x7efd640cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2937761726 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6406d300 0x7efd640cfd90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2937761726 >> 172.21.15.16:0/2937761726 conn(0x7efd6408a270 msgr2=0x7efd64099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2937761726 shutdown_connections 2023-10-06T11:43:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.842+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2937761726 wait complete. 2023-10-06T11:43:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.843+0000 7efd6b4f0700 1 Processor -- start 2023-10-06T11:43:18.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.843+0000 7efd6b4f0700 1 -- start start 2023-10-06T11:43:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 0x7efd64152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 0x7efd640b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640c8560 con 0x7efd640cd190 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640c7970 con 0x7efd6415ab50 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6b4f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640c7c30 con 0x7efd6406d300 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd68a8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd68a8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38294/0 (socket says 172.21.15.16:38294) 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd68a8b700 1 -- 172.21.15.16:0/2089147244 learned_addr learned my addr 172.21.15.16:0/2089147244 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd69a8d700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 0x7efd640b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd6928c700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 0x7efd64152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.844+0000 7efd68a8b700 1 -- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 msgr2=0x7efd64152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd68a8b700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 0x7efd64152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd68a8b700 1 -- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 msgr2=0x7efd640b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd68a8b700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 0x7efd640b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd68a8b700 1 -- 172.21.15.16:0/2089147244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd580094f0 con 0x7efd640cd190 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd6928c700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 0x7efd64152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd69a8d700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 0x7efd640b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd68a8b700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd64064a90 tx=0x7efd58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800bbe0 con 0x7efd640cd190 2023-10-06T11:43:18.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58011420 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd640c7f10 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd580117e0 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.845+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd64153380 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.846+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd640c8710 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.846+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7efd58011b50 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.847+0000 7efd617fa700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 0x7efd500695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.847+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7efd58085b30 con 0x7efd640cd190 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.847+0000 7efd6928c700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 0x7efd500695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:18.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.847+0000 7efd6928c700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 0x7efd500695c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd540015b0 tx=0x7efd54009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:18.851+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd640c8710 con 0x7efd640cd190 2023-10-06T11:43:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.042+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7efd64063e70 con 0x7efd50067110 2023-10-06T11:43:19.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.042+0000 7efd617fa700 1 -- 172.21.15.16:0/2089147244 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efd64063e70 con 0x7efd50067110 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 msgr2=0x7efd500695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 0x7efd500695c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd540015b0 tx=0x7efd54009040 comp rx=0 tx=0).stop 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 msgr2=0x7efd640b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efd64064a90 tx=0x7efd58017000 comp rx=0 tx=0).stop 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 shutdown_connections 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7efd50067110 0x7efd500695c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.044+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6415ab50 0x7efd640b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.045+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd6406d300 0x7efd64152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.045+0000 7efd6b4f0700 1 --2- 172.21.15.16:0/2089147244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd640cd190 0x7efd640b5250 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.045+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 >> 172.21.15.16:0/2089147244 conn(0x7efd6408a270 msgr2=0x7efd640635e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.045+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 shutdown_connections 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.045+0000 7efd6b4f0700 1 -- 172.21.15.16:0/2089147244 wait complete. 2023-10-06T11:43:19.047 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T11:43:19.058 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T11:43:19.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:19.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.181+0000 7f43d60ef700 1 Processor -- start 2023-10-06T11:43:19.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.181+0000 7f43d60ef700 1 -- start start 2023-10-06T11:43:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43d60ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43d00b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43d60ef700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d015a760 con 0x7f43d0153fc0 2023-10-06T11:43:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43d60ef700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d00b2090 con 0x7f43d0153bf0 2023-10-06T11:43:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43d60ef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0159e30 con 0x7f43d00ceb90 2023-10-06T11:43:19.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43ceffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f43d0153fc0 0x7f43d00b1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59042/0 (socket says 172.21.15.16:59042) 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.182+0000 7f43ceffd700 1 -- 172.21.15.16:0/2224919362 learned_addr learned my addr 172.21.15.16:0/2224919362 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43cffff700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43d00b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208359719 0 0) 0x7f43d015a760 con 0x7f43d0153fc0 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b8003570 con 0x7f43d0153fc0 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1048391580 0 0) 0x7f43b8003570 con 0x7f43d0153fc0 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 msgr2=0x7f43d00b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43d00b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 >> v1:172.21.15.177:6789/0 conn(0x7f43d0153bf0 legacy=0x7f43d00ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d015a110 con 0x7f43d0153fc0 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f43c4001470 con 0x7f43d0153fc0 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 >> v1:172.21.15.16:6789/0 conn(0x7f43d0153fc0 legacy=0x7f43d00b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43b8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8004010 0x7f43b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004d50 0x7f43b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0159e30 con 0x7f43b8004010 2023-10-06T11:43:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b800bbf0 con 0x7f43b8004d50 2023-10-06T11:43:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b800bdb0 con 0x7f43d00ceb90 2023-10-06T11:43:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8004010 0x7f43b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43cffff700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004d50 0x7f43b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43cf7fe700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43b8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 -- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 msgr2=0x7f43b8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43b8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 -- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004d50 msgr2=0x7f43b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004d50 0x7f43b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 -- 172.21.15.16:0/2224919362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8003570 con 0x7f43b8004010 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43cffff700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004d50 0x7f43b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43cf7fe700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d00ceb90 0x7f43b8003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.183+0000 7f43ceffd700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8004010 0x7f43b8004810 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001ed0 tx=0x7f43d0159d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb62a0b6a2c3ede5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400cc30 con 0x7f43b8004010 2023-10-06T11:43:19.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c400cd90 con 0x7f43b8004010 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43ce7fc700 1 -- 172.21.15.16:0/2224919362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4005b70 con 0x7f43b8004010 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 -- 172.21.15.16:0/2224919362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8004010 msgr2=0x7f43b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 --2- 172.21.15.16:0/2224919362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8004010 0x7f43b8004810 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001ed0 tx=0x7f43d0159d20 comp rx=0 tx=0).stop 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43cffff700 1 -- 172.21.15.16:0/2224919362 reap_dead start 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 -- 172.21.15.16:0/2224919362 shutdown_connections 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 -- 172.21.15.16:0/2224919362 >> 172.21.15.16:0/2224919362 conn(0x7f43d006cee0 msgr2=0x7f43d0067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 -- 172.21.15.16:0/2224919362 shutdown_connections 2023-10-06T11:43:19.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.184+0000 7f43d60ef700 1 -- 172.21.15.16:0/2224919362 wait complete. 2023-10-06T11:43:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.185+0000 7f43d60ef700 1 Processor -- start 2023-10-06T11:43:19.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.185+0000 7f43d60ef700 1 -- start start 2023-10-06T11:43:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004010 0x7f43d013def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 0x7f43d015d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d014f820 0x7f43d014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01501e0 con 0x7f43d015d3b0 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d01517d0 con 0x7f43b8004010 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43d60ef700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0151a70 con 0x7f43d014f820 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 0x7f43d015d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 0x7f43d015d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38318/0 (socket says 172.21.15.16:38318) 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43ceffd700 1 -- 172.21.15.16:0/4222119321 learned_addr learned my addr 172.21.15.16:0/4222119321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43cffff700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d014f820 0x7f43d014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.186+0000 7f43cf7fe700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004010 0x7f43d013def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 -- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d014f820 msgr2=0x7f43d014fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f43d014f820 0x7f43d014fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 -- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004010 msgr2=0x7f43d013def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43b8004010 0x7f43d013def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0151d50 con 0x7f43d015d3b0 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43ceffd700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 0x7f43d015d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f43b80057d0 tx=0x7f43c4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4001740 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40070d0 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d0152030 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4007490 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.187+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d01525f0 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.188+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f43d0063e70 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.188+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f43c4019030 con 0x7f43d015d3b0 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43b77fe700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f43bc0670e0 0x7f43bc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f43c4085660 con 0x7f43d015d3b0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43b77fe700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f43bc06d550 0x7f43bc06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43bc070030 con 0x7f43bc06d550 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43c4087c80 con 0x7f43d015d3b0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43cf7fe700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f43bc0670e0 0x7f43bc069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43cffff700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f43bc06d550 0x7f43bc06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.189+0000 7f43cf7fe700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f43bc0670e0 0x7f43bc069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43c0007fd0 tx=0x7f43c0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.190+0000 7f43cffff700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f43bc06d550 0x7f43bc06f960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.190+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== osd.5 v2:172.21.15.177:6800/90843 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f43bc070030 con 0x7f43bc06d550 2023-10-06T11:43:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.204+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 --> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f43d015a3f0 con 0x7f43bc06d550 2023-10-06T11:43:19.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43b77fe700 1 -- 172.21.15.16:0/4222119321 <== osd.5 v2:172.21.15.177:6800/90843 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f43d015a3f0 con 0x7f43bc06d550 2023-10-06T11:43:19.206 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:19.206 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:19.206 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f43bc06d550 msgr2=0x7f43bc06f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6800/90843,v1:172.21.15.177:6801/90843] conn(0x7f43bc06d550 0x7f43bc06f960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f43bc0670e0 msgr2=0x7f43bc069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f43bc0670e0 0x7f43bc069590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43c0007fd0 tx=0x7f43c0009040 comp rx=0 tx=0).stop 2023-10-06T11:43:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 msgr2=0x7f43d015d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 --2- 172.21.15.16:0/4222119321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43d015d3b0 0x7f43d015d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f43b80057d0 tx=0x7f43c4015000 comp rx=0 tx=0).stop 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43cffff700 1 -- 172.21.15.16:0/4222119321 reap_dead start 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 shutdown_connections 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 >> 172.21.15.16:0/4222119321 conn(0x7f43d006cee0 msgr2=0x7f43d0067dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 shutdown_connections 2023-10-06T11:43:19.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.205+0000 7f43d60ef700 1 -- 172.21.15.16:0/4222119321 wait complete. 2023-10-06T11:43:19.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T11:43:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.337+0000 7fd9e0aed700 1 Processor -- start 2023-10-06T11:43:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.337+0000 7fd9e0aed700 1 -- start start 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9e0aed700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd9dc15d920 0x7fd9dc0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9e0aed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc15a650 con 0x7fd9dc159790 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9e0aed700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9dc0b61c0 con 0x7fd9dc1593c0 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9e0aed700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9dc159d20 con 0x7fd9dc15d920 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9dbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9dc159790 0x7fd9dc0b5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59048/0 (socket says 172.21.15.16:59048) 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9dbfff700 1 -- 172.21.15.16:0/1156499800 learned_addr learned my addr 172.21.15.16:0/1156499800 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9daffd700 1 --2- 172.21.15.16:0/1156499800 >> v2:172.21.15.177:3300/0 conn(0x7fd9dc15d920 0x7fd9dc0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113243248 0 0) 0x7fd9dc15a650 con 0x7fd9dc159790 2023-10-06T11:43:19.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.338+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c4003570 con 0x7fd9dc159790 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1105153837 0 0) 0x7fd9c4003570 con 0x7fd9dc159790 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 >> v2:172.21.15.177:3300/0 conn(0x7fd9dc15d920 msgr2=0x7fd9dc0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 --2- 172.21.15.16:0/1156499800 >> v2:172.21.15.177:3300/0 conn(0x7fd9dc15d920 0x7fd9dc0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 >> v1:172.21.15.177:6789/0 conn(0x7fd9dc1593c0 legacy=0x7fd9dc15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc15a000 con 0x7fd9dc159790 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7fd9d40017f0 con 0x7fd9dc159790 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 >> v1:172.21.15.16:6789/0 conn(0x7fd9dc159790 legacy=0x7fd9dc0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9c4004810 0x7fd9c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9c40051c0 0x7fd9c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc159d20 con 0x7fd9c4003a60 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400c0d0 con 0x7fd9c4004810 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c400c290 con 0x7fd9c40051c0 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9daffd700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9c4004810 0x7fd9c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9dbfff700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9c40051c0 0x7fd9c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 -- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9c40051c0 msgr2=0x7fd9c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9c40051c0 0x7fd9c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 -- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9c4004810 msgr2=0x7fd9c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9c4004810 0x7fd9c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 -- 172.21.15.16:0/1156499800 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c4003570 con 0x7fd9c4003a60 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9dbfff700 1 -- 172.21.15.16:0/1156499800 reap_dead start 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.339+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9c4003e70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0018d0 tx=0x7fd9cc007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65515f13ed2e67f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc001290 con 0x7fd9c4003a60 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc012420 con 0x7fd9c4003a60 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9da7fc700 1 -- 172.21.15.16:0/1156499800 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc0127e0 con 0x7fd9c4003a60 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 -- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 msgr2=0x7fd9c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9c4003e70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0018d0 tx=0x7fd9cc007d80 comp rx=0 tx=0).stop 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 -- 172.21.15.16:0/1156499800 shutdown_connections 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/1156499800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9c4003e70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 -- 172.21.15.16:0/1156499800 >> 172.21.15.16:0/1156499800 conn(0x7fd9dc08a020 msgr2=0x7fd9dc099940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 -- 172.21.15.16:0/1156499800 shutdown_connections 2023-10-06T11:43:19.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.340+0000 7fd9e0aed700 1 -- 172.21.15.16:0/1156499800 wait complete. 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.341+0000 7fd9e0aed700 1 Processor -- start 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.341+0000 7fd9e0aed700 1 -- start start 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 0x7fd9dc149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9dc14dd90 0x7fd9dc151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc150d30 con 0x7fd9c4003a60 2023-10-06T11:43:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc1518b0 con 0x7fd9dc150680 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9e0aed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9dc150af0 con 0x7fd9dc14dd90 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38334/0 (socket says 172.21.15.16:38334) 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9db7fe700 1 -- 172.21.15.16:0/899832116 learned_addr learned my addr 172.21.15.16:0/899832116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.342+0000 7fd9daffd700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 0x7fd9dc149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9dbfff700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9dc14dd90 0x7fd9dc151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 -- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9dc14dd90 msgr2=0x7fd9dc151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9dc14dd90 0x7fd9dc151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 -- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 msgr2=0x7fd9dc149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 0x7fd9dc149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 -- 172.21.15.16:0/899832116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9dc150f80 con 0x7fd9c4003a60 2023-10-06T11:43:19.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9daffd700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 0x7fd9dc149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9db7fe700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc001060 tx=0x7fd9cc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc006a80 con 0x7fd9c4003a60 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc006c20 con 0x7fd9c4003a60 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc151260 con 0x7fd9c4003a60 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc007030 con 0x7fd9c4003a60 2023-10-06T11:43:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.343+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9dc149b50 con 0x7fd9c4003a60 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.344+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9dc151b90 con 0x7fd9c4003a60 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.345+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd9cc019070 con 0x7fd9c4003a60 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.345+0000 7fd9c37fe700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 0x7fd9c80695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.345+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd9cc085970 con 0x7fd9c4003a60 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.345+0000 7fd9daffd700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 0x7fd9c80695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.346+0000 7fd9daffd700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 0x7fd9c80695c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006d00 tx=0x7fd9d0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.349+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9cc052b00 con 0x7fd9c4003a60 2023-10-06T11:43:19.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.542+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9dc150d30 con 0x7fd9c8067110 2023-10-06T11:43:19.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.542+0000 7fd9c37fe700 1 -- 172.21.15.16:0/899832116 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd9dc150d30 con 0x7fd9c8067110 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 msgr2=0x7fd9c80695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 0x7fd9c80695c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0006d00 tx=0x7fd9d0008040 comp rx=0 tx=0).stop 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 msgr2=0x7fd9dc14a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc001060 tx=0x7fd9cc015000 comp rx=0 tx=0).stop 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 shutdown_connections 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c4003a60 0x7fd9dc14a210 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd9c8067110 0x7fd9c80695c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9dc14dd90 0x7fd9dc151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 --2- 172.21.15.16:0/899832116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9dc150680 0x7fd9dc149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 >> 172.21.15.16:0/899832116 conn(0x7fd9dc08a020 msgr2=0x7fd9dc06efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 shutdown_connections 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.545+0000 7fd9e0aed700 1 -- 172.21.15.16:0/899832116 wait complete. 2023-10-06T11:43:19.547 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T11:43:19.558 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T11:43:19.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.680+0000 7f7de913e700 1 Processor -- start 2023-10-06T11:43:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.680+0000 7f7de913e700 1 -- start start 2023-10-06T11:43:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de913e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7de40b4850 0x7f7de40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de913e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de40b33b0 con 0x7f7de40b6030 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de913e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de4159a70 con 0x7f7de40d04e0 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de913e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de40b6490 con 0x7f7de40b4850 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7de40b4850 0x7f7de40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de893d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7de40b6030 0x7f7de4163230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59062/0 (socket says 172.21.15.16:59062) 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de893d700 1 -- 172.21.15.16:0/4135900265 learned_addr learned my addr 172.21.15.16:0/4135900265 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de37fe700 1 -- 172.21.15.16:0/4135900265 >> v1:172.21.15.177:6789/0 conn(0x7f7de40d04e0 legacy=0x7f7de40d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.681+0000 7f7de37fe700 1 -- 172.21.15.16:0/4135900265 >> v1:172.21.15.16:6789/0 conn(0x7f7de40b6030 legacy=0x7f7de4163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de37fe700 1 -- 172.21.15.16:0/4135900265 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40cf800 con 0x7f7de40b4850 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de2ffd700 1 -- 172.21.15.16:0/4135900265 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053666226 0 0) 0x7f7de40b33b0 con 0x7f7de40b6030 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de37fe700 1 --2- 172.21.15.16:0/4135900265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b4850 0x7f7de40ccff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0098e0 tx=0x7f7dcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87af4e59edffb71 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de2ffd700 1 -- 172.21.15.16:0/4135900265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc009e80 con 0x7f7de40b4850 2023-10-06T11:43:19.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de2ffd700 1 -- 172.21.15.16:0/4135900265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc00b860 con 0x7f7de40b4850 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de2ffd700 1 -- 172.21.15.16:0/4135900265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00bbe0 con 0x7f7de40b4850 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 -- 172.21.15.16:0/4135900265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b4850 msgr2=0x7f7de40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 --2- 172.21.15.16:0/4135900265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b4850 0x7f7de40ccff0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc0098e0 tx=0x7f7dcc0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 -- 172.21.15.16:0/4135900265 shutdown_connections 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 --2- 172.21.15.16:0/4135900265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b4850 0x7f7de40ccff0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 -- 172.21.15.16:0/4135900265 >> 172.21.15.16:0/4135900265 conn(0x7f7de406cea0 msgr2=0x7f7de406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 -- 172.21.15.16:0/4135900265 shutdown_connections 2023-10-06T11:43:19.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.682+0000 7f7de913e700 1 -- 172.21.15.16:0/4135900265 wait complete. 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 Processor -- start 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 -- start start 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7de40b4850 0x7f7de4144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 0x7f7de40b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7de40d04e0 0x7f7de40b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4144be0 con 0x7f7de40b6030 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de41460d0 con 0x7f7de40d04e0 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.684+0000 7f7de913e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de41454c0 con 0x7f7de40b4850 2023-10-06T11:43:19.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 0x7f7de40b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 0x7f7de40b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38350/0 (socket says 172.21.15.16:38350) 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 -- 172.21.15.16:0/2699399253 learned_addr learned my addr 172.21.15.16:0/2699399253 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de893d700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7de40d04e0 0x7f7de40b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 -- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7de40b4850 msgr2=0x7f7de4144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de3fff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7de40b4850 0x7f7de4144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7de40b4850 0x7f7de4144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 -- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7de40d04e0 msgr2=0x7f7de40b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7de40d04e0 0x7f7de40b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc009420 con 0x7f7de40b6030 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de893d700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7de40d04e0 0x7f7de40b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de3fff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7de40b4850 0x7f7de4144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de37fe700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 0x7f7de40b1e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7de40b3070 tx=0x7f7dcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc01f070 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc011420 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de41457a0 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00bbe0 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.685+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de4145d60 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.686+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7de404c380 con 0x7f7de40b6030 2023-10-06T11:43:19.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.687+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7dcc011a50 con 0x7f7de40b6030 2023-10-06T11:43:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.687+0000 7f7ddbfff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7dc4067110 0x7f7dc40695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.687+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7dcc086220 con 0x7f7de40b6030 2023-10-06T11:43:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7ddbfff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f7dc406d580 0x7f7dc406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7dc4070060 con 0x7f7dc406d580 2023-10-06T11:43:19.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dcc086510 con 0x7f7de40b6030 2023-10-06T11:43:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7de3fff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7dc4067110 0x7f7dc40695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7de893d700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f7dc406d580 0x7f7dc406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7de3fff700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7dc4067110 0x7f7dc40695c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40017a0 tx=0x7f7dd4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.688+0000 7f7de893d700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f7dc406d580 0x7f7dc406f990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.689+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== osd.6 v2:172.21.15.177:6824/90887 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7dc4070060 con 0x7f7dc406d580 2023-10-06T11:43:19.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 --> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7de4159180 con 0x7f7dc406d580 2023-10-06T11:43:19.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7ddbfff700 1 -- 172.21.15.16:0/2699399253 <== osd.6 v2:172.21.15.177:6824/90887 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7de4159180 con 0x7f7dc406d580 2023-10-06T11:43:19.704 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:19.704 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:19.704 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:19.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f7dc406d580 msgr2=0x7f7dc406f990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6824/90887,v1:172.21.15.177:6825/90887] conn(0x7f7dc406d580 0x7f7dc406f990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7dc4067110 msgr2=0x7f7dc40695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7dc4067110 0x7f7dc40695c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40017a0 tx=0x7f7dd4009040 comp rx=0 tx=0).stop 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 msgr2=0x7f7de40b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 --2- 172.21.15.16:0/2699399253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7de40b6030 0x7f7de40b1e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7de40b3070 tx=0x7f7dcc017000 comp rx=0 tx=0).stop 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de893d700 1 -- 172.21.15.16:0/2699399253 reap_dead start 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 shutdown_connections 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.703+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 >> 172.21.15.16:0/2699399253 conn(0x7f7de406cea0 msgr2=0x7f7de40b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.704+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 shutdown_connections 2023-10-06T11:43:19.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.704+0000 7f7de913e700 1 -- 172.21.15.16:0/2699399253 wait complete. 2023-10-06T11:43:19.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T11:43:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.835+0000 7f103359e700 1 Processor -- start 2023-10-06T11:43:19.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.835+0000 7f103359e700 1 -- start start 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 0x7f103406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103359e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1034155260 0x7f103415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103359e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10340b0d80 con 0x7f1034158c80 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103359e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103415c080 con 0x7f1034155260 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103359e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340b01b0 con 0x7f1034154e90 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1034155260 0x7f103415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1031d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1034158c80 0x7f103415b350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59066/0 (socket says 172.21.15.16:59066) 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1034155260 0x7f103415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38354/0 (socket says 172.21.15.16:38354) 2023-10-06T11:43:19.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1031d9b700 1 -- 172.21.15.16:0/2826335033 learned_addr learned my addr 172.21.15.16:0/2826335033 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103259c700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 0x7f103406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 -- 172.21.15.16:0/2826335033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 msgr2=0x7f103406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 0x7f103406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 -- 172.21.15.16:0/2826335033 >> v1:172.21.15.16:6789/0 conn(0x7f1034158c80 legacy=0x7f103415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f103159a700 1 -- 172.21.15.16:0/2826335033 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907335173 0 0) 0x7f10340b0d80 con 0x7f1034158c80 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.836+0000 7f1032d9d700 1 -- 172.21.15.16:0/2826335033 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340b0450 con 0x7f1034155260 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103259c700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 0x7f103406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f1032d9d700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034155260 0x7f103415ba60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1024009e60 tx=0x7f1024009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b49410abf7786f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103159a700 1 -- 172.21.15.16:0/2826335033 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400e070 con 0x7f1034155260 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103159a700 1 -- 172.21.15.16:0/2826335033 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102400bc30 con 0x7f1034155260 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103159a700 1 -- 172.21.15.16:0/2826335033 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024012430 con 0x7f1034155260 2023-10-06T11:43:19.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 -- 172.21.15.16:0/2826335033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034155260 msgr2=0x7f103415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034155260 0x7f103415ba60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1024009e60 tx=0x7f1024009780 comp rx=0 tx=0).stop 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 -- 172.21.15.16:0/2826335033 shutdown_connections 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034155260 0x7f103415ba60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 --2- 172.21.15.16:0/2826335033 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034154e90 0x7f103406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 -- 172.21.15.16:0/2826335033 >> 172.21.15.16:0/2826335033 conn(0x7f103406c410 msgr2=0x7f103406c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 -- 172.21.15.16:0/2826335033 shutdown_connections 2023-10-06T11:43:19.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.837+0000 7f103359e700 1 -- 172.21.15.16:0/2826335033 wait complete. 2023-10-06T11:43:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 Processor -- start 2023-10-06T11:43:19.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 -- start start 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034155260 0x7f1034159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1034158c80 0x7f10341595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341687a0 con 0x7f1034154e90 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1034148930 con 0x7f1034158c80 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103359e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103414fb80 con 0x7f1034155260 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38368/0 (socket says 172.21.15.16:38368) 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.839+0000 7f103259c700 1 -- 172.21.15.16:0/4095240661 learned_addr learned my addr 172.21.15.16:0/4095240661 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f1032d9d700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1034158c80 0x7f10341595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 -- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034155260 msgr2=0x7f1034159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034155260 0x7f1034159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 -- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1034158c80 msgr2=0x7f10341595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1034158c80 0x7f10341595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 -- 172.21.15.16:0/4095240661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024009a20 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103259c700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f102c00c830 tx=0x7f102c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011600 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102c011760 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102c011b20 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10341684c0 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.840+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1034150120 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.841+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1034150340 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.841+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f102c00a3e0 con 0x7f1034154e90 2023-10-06T11:43:19.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.842+0000 7f102a7fc700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 0x7f1014069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.842+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f102c084c70 con 0x7f1034154e90 2023-10-06T11:43:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.842+0000 7f1031d9b700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 0x7f1014069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:19.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.842+0000 7f1031d9b700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 0x7f1014069590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1034159c60 tx=0x7f101c007bd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:19.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:19.846+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f102c051cf0 con 0x7f1034154e90 2023-10-06T11:43:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.039+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f103404c380 con 0x7f10140670e0 2023-10-06T11:43:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.040+0000 7f102a7fc700 1 -- 172.21.15.16:0/4095240661 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f103404c380 con 0x7f10140670e0 2023-10-06T11:43:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 msgr2=0x7f1014069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 0x7f1014069590 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1034159c60 tx=0x7f101c007bd0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 msgr2=0x7f1034147c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f102c00c830 tx=0x7f102c008040 comp rx=0 tx=0).stop 2023-10-06T11:43:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 shutdown_connections 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1034154e90 0x7f1034147c40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f10140670e0 0x7f1014069590 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1034158c80 0x7f10341595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 --2- 172.21.15.16:0/4095240661 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1034155260 0x7f1034159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 >> 172.21.15.16:0/4095240661 conn(0x7f103406c410 msgr2=0x7f1034152f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 shutdown_connections 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.042+0000 7f103359e700 1 -- 172.21.15.16:0/4095240661 wait complete. 2023-10-06T11:43:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T11:43:20.055 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T11:43:20.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T11:43:20.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 Processor -- start 2023-10-06T11:43:20.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 -- start start 2023-10-06T11:43:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f27780ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f27780b6170 0x7f27780b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27780b2630 con 0x7f27780b3fc0 2023-10-06T11:43:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27780b65e0 con 0x7f27780b6170 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.177+0000 7f277e3c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27780b1a80 con 0x7f2778159810 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f277c962700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27780b3fc0 0x7f27780b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59080/0 (socket says 172.21.15.16:59080) 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f277c962700 1 -- 172.21.15.16:0/4173909759 learned_addr learned my addr 172.21.15.16:0/4173909759 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f27777fe700 1 --2- 172.21.15.16:0/4173909759 >> v2:172.21.15.177:3300/0 conn(0x7f27780b6170 0x7f27780b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2777fff700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f27780ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730488651 0 0) 0x7f27780b2630 con 0x7f27780b3fc0 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2760003570 con 0x7f27780b3fc0 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3229620463 0 0) 0x7f2760003570 con 0x7f27780b3fc0 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 msgr2=0x7f27780ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f27780ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 >> v2:172.21.15.177:3300/0 conn(0x7f27780b6170 msgr2=0x7f27780b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 --2- 172.21.15.16:0/4173909759 >> v2:172.21.15.177:3300/0 conn(0x7f27780b6170 0x7f27780b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780b1d00 con 0x7f27780b3fc0 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f27700017f0 con 0x7f27780b3fc0 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 >> v1:172.21.15.16:6789/0 conn(0x7f27780b3fc0 legacy=0x7f27780b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f2760003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2760004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2760004d50 0x7f27600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780b65e0 con 0x7f2760003fe0 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780b1a80 con 0x7f2760004d50 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276000ba90 con 0x7f2778159810 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f27777fe700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2760004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f2777fff700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f2760003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.178+0000 7f277c962700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2760004d50 0x7f27600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 -- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 msgr2=0x7f2760003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2778159810 0x7f2760003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 -- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2760004d50 msgr2=0x7f27600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2760004d50 0x7f27600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 -- 172.21.15.16:0/4173909759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760003570 con 0x7f2760003fe0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f27777fe700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2760004810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f276c009c00 tx=0x7f276c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba556d4393b4bf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00e6b0 con 0x7f2760003fe0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c00e810 con 0x7f2760003fe0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.179+0000 7f2776ffd700 1 -- 172.21.15.16:0/4173909759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00ebd0 con 0x7f2760003fe0 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 -- 172.21.15.16:0/4173909759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 msgr2=0x7f2760004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 --2- 172.21.15.16:0/4173909759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2760004810 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f276c009c00 tx=0x7f276c0091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277c962700 1 -- 172.21.15.16:0/4173909759 reap_dead start 2023-10-06T11:43:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 -- 172.21.15.16:0/4173909759 shutdown_connections 2023-10-06T11:43:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 -- 172.21.15.16:0/4173909759 >> 172.21.15.16:0/4173909759 conn(0x7f277808a1e0 msgr2=0x7f2778099b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 -- 172.21.15.16:0/4173909759 shutdown_connections 2023-10-06T11:43:20.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.180+0000 7f277e3c5700 1 -- 172.21.15.16:0/4173909759 wait complete. 2023-10-06T11:43:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.181+0000 7f277e3c5700 1 Processor -- start 2023-10-06T11:43:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.181+0000 7f277e3c5700 1 -- start start 2023-10-06T11:43:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2778156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27780ab810 0x7f2778159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27780abc80 0x7f277815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277809b280 con 0x7f2760003fe0 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781572f0 con 0x7f27780abc80 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27780aeec0 con 0x7f27780ab810 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2778156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2778156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38380/0 (socket says 172.21.15.16:38380) 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 -- 172.21.15.16:0/4236880513 learned_addr learned my addr 172.21.15.16:0/4236880513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f27777fe700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27780ab810 0x7f2778159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277c962700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27780abc80 0x7f277815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 -- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27780ab810 msgr2=0x7f2778159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f27780ab810 0x7f2778159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 -- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27780abc80 msgr2=0x7f277815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27780abc80 0x7f277815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277809af50 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277c962700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f27780abc80 0x7f277815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f2777fff700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2778156db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2768011d40 tx=0x7f27680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800fac0 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276800fc20 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780af160 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768014440 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.182+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780af720 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.183+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2778063e70 con 0x7f2760003fe0 2023-10-06T11:43:20.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f276801c3e0 con 0x7f2760003fe0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f27640670e0 0x7f2764069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f2768018030 con 0x7f2760003fe0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7f276406d550 0x7f276406f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2764070030 con 0x7f276406d550 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27680857e0 con 0x7f2760003fe0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.184+0000 7f27777fe700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f27640670e0 0x7f2764069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.185+0000 7f277c962700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7f276406d550 0x7f276406f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.185+0000 7f27777fe700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f27640670e0 0x7f2764069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f276c0091e0 tx=0x7f276c01a000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.185+0000 7f277c962700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7f276406d550 0x7f276406f960 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.186+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== osd.7 v2:172.21.15.177:6816/90844 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2764070030 con 0x7f276406d550 2023-10-06T11:43:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.200+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 --> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f27780b1fe0 con 0x7f276406d550 2023-10-06T11:43:20.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.200+0000 7f275ffff700 1 -- 172.21.15.16:0/4236880513 <== osd.7 v2:172.21.15.177:6816/90844 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f27780b1fe0 con 0x7f276406d550 2023-10-06T11:43:20.202 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-10-06T11:43:20.202 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T11:43:20.202 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7f276406d550 msgr2=0x7f276406f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6816/90844,v1:172.21.15.177:6817/90844] conn(0x7f276406d550 0x7f276406f960 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f27640670e0 msgr2=0x7f2764069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f27640670e0 0x7f2764069590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f276c0091e0 tx=0x7f276c01a000 comp rx=0 tx=0).stop 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 msgr2=0x7f2778156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 --2- 172.21.15.16:0/4236880513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2760003fe0 0x7f2778156db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2768011d40 tx=0x7f27680091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277c962700 1 -- 172.21.15.16:0/4236880513 reap_dead start 2023-10-06T11:43:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 shutdown_connections 2023-10-06T11:43:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 >> 172.21.15.16:0/4236880513 conn(0x7f277808a1e0 msgr2=0x7f2778158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 shutdown_connections 2023-10-06T11:43:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.201+0000 7f277e3c5700 1 -- 172.21.15.16:0/4236880513 wait complete. 2023-10-06T11:43:20.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T11:43:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 Processor -- start 2023-10-06T11:43:20.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 -- start start 2023-10-06T11:43:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f064006cbf0 0x7f064006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f06400cd190 0x7f06400cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0640154580 con 0x7f064006cbf0 2023-10-06T11:43:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064006d5a0 con 0x7f06400cd190 2023-10-06T11:43:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.333+0000 7f0645de9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06400cf060 con 0x7f064015ab50 2023-10-06T11:43:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f064006cbf0 0x7f064006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f064006cbf0 0x7f064006d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38394/0 (socket says 172.21.15.16:38394) 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 -- 172.21.15.16:0/929422157 learned_addr learned my addr 172.21.15.16:0/929422157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063ffff700 1 --2- 172.21.15.16:0/929422157 >> v2:172.21.15.177:3300/0 conn(0x7f06400cd190 0x7f06400cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063f7fe700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 -- 172.21.15.16:0/929422157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 msgr2=0x7f06400cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 -- 172.21.15.16:0/929422157 >> v2:172.21.15.177:3300/0 conn(0x7f06400cd190 msgr2=0x7f06400cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 --2- 172.21.15.16:0/929422157 >> v2:172.21.15.177:3300/0 conn(0x7f06400cd190 0x7f06400cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 -- 172.21.15.16:0/929422157 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400cf900 con 0x7f064006cbf0 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063effd700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064006cbf0 0x7f064006d060 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06400c3e80 tx=0x7f0630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d510aa101ddac3aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063e7fc700 1 -- 172.21.15.16:0/929422157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000b820 con 0x7f064006cbf0 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063e7fc700 1 -- 172.21.15.16:0/929422157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063000b980 con 0x7f064006cbf0 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.334+0000 7f063e7fc700 1 -- 172.21.15.16:0/929422157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000bd20 con 0x7f064006cbf0 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 -- 172.21.15.16:0/929422157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064006cbf0 msgr2=0x7f064006d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064006cbf0 0x7f064006d060 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f06400c3e80 tx=0x7f0630009250 comp rx=0 tx=0).stop 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 -- 172.21.15.16:0/929422157 shutdown_connections 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 --2- 172.21.15.16:0/929422157 >> v2:172.21.15.177:3300/0 conn(0x7f06400cd190 0x7f06400cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 --2- 172.21.15.16:0/929422157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f064006cbf0 0x7f064006d060 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 -- 172.21.15.16:0/929422157 >> 172.21.15.16:0/929422157 conn(0x7f064008a270 msgr2=0x7f0640099b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 -- 172.21.15.16:0/929422157 shutdown_connections 2023-10-06T11:43:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.335+0000 7f0645de9700 1 -- 172.21.15.16:0/929422157 wait complete. 2023-10-06T11:43:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.336+0000 7f0645de9700 1 Processor -- start 2023-10-06T11:43:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.336+0000 7f0645de9700 1 -- start start 2023-10-06T11:43:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f064006cbf0 0x7f0640155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401572b0 con 0x7f06400cd190 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401566a0 con 0x7f064006cbf0 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f0645de9700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0640156980 con 0x7f064015ab50 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38404/0 (socket says 172.21.15.16:38404) 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 -- 172.21.15.16:0/1850177053 learned_addr learned my addr 172.21.15.16:0/1850177053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063f7fe700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f064006cbf0 0x7f0640155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 -- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 msgr2=0x7f06400b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 -- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f064006cbf0 msgr2=0x7f0640155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f064006cbf0 0x7f0640155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.337+0000 7f063effd700 1 -- 172.21.15.16:0/1850177053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300094f0 con 0x7f06400cd190 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f063effd700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06300094c0 tx=0x7f0630017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630011e20 con 0x7f06400cd190 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0630024440 con 0x7f06400cd190 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630024800 con 0x7f06400cd190 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0640156c60 con 0x7f06400cd190 2023-10-06T11:43:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.338+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064015d5b0 con 0x7f06400cd190 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.339+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064015d7a0 con 0x7f06400cd190 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.339+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f063001a420 con 0x7f06400cd190 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.339+0000 7f06277fe700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 0x7f062c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.340+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f0630086590 con 0x7f06400cd190 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.340+0000 7f063f7fe700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 0x7f062c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.340+0000 7f063f7fe700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 0x7f062c069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0634006aa0 tx=0x7f0634008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.344+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f064015d7a0 con 0x7f06400cd190 2023-10-06T11:43:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.537+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f064015d7a0 con 0x7f062c0670e0 2023-10-06T11:43:20.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.537+0000 7f06277fe700 1 -- 172.21.15.16:0/1850177053 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f064015d7a0 con 0x7f062c0670e0 2023-10-06T11:43:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 msgr2=0x7f062c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 0x7f062c069590 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0634006aa0 tx=0x7f0634008040 comp rx=0 tx=0).stop 2023-10-06T11:43:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 msgr2=0x7f06400b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f06300094c0 tx=0x7f0630017000 comp rx=0 tx=0).stop 2023-10-06T11:43:20.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 shutdown_connections 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f062c0670e0 0x7f062c069590 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f064015ab50 0x7f06400b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f064006cbf0 0x7f0640155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 --2- 172.21.15.16:0/1850177053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06400cd190 0x7f06400b3590 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 >> 172.21.15.16:0/1850177053 conn(0x7f064008a270 msgr2=0x7f0640151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 shutdown_connections 2023-10-06T11:43:20.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.540+0000 7f0645de9700 1 -- 172.21.15.16:0/1850177053 wait complete. 2023-10-06T11:43:20.543 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T11:43:20.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:43:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.671+0000 7f05d8915700 1 Processor -- start 2023-10-06T11:43:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.671+0000 7f05d8915700 1 -- start start 2023-10-06T11:43:20.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d8915700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f05d4156550 0x7f05d4069270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d8915700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d406ad10 0x7f05d406b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d8915700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05d40b2210 con 0x7f05d40b0ea0 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d8915700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d406b680 con 0x7f05d406ad10 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d8915700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05d40b1600 con 0x7f05d4156550 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d406ad10 0x7f05d406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05d406ad10 0x7f05d406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38418/0 (socket says 172.21.15.16:38418) 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 -- 172.21.15.16:0/1869504725 learned_addr learned my addr 172.21.15.16:0/1869504725 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d259c700 1 --2- 172.21.15.16:0/1869504725 >> v2:172.21.15.177:3300/0 conn(0x7f05d4156550 0x7f05d4069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 -- 172.21.15.16:0/1869504725 >> v2:172.21.15.177:3300/0 conn(0x7f05d4156550 msgr2=0x7f05d4069270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 --2- 172.21.15.16:0/1869504725 >> v2:172.21.15.177:3300/0 conn(0x7f05d4156550 0x7f05d4069270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 -- 172.21.15.16:0/1869504725 >> v1:172.21.15.177:6789/0 conn(0x7f05d40b0ea0 legacy=0x7f05d4156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.672+0000 7f05d2d9d700 1 -- 172.21.15.16:0/1869504725 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d40b18e0 con 0x7f05d406ad10 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d2d9d700 1 --2- 172.21.15.16:0/1869504725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d406b140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f05d406a720 tx=0x7f05cc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f10119ed3d1232 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d159a700 1 -- 172.21.15.16:0/1869504725 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00dc10 con 0x7f05d406ad10 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d159a700 1 -- 172.21.15.16:0/1869504725 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc00dd70 con 0x7f05d406ad10 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d159a700 1 -- 172.21.15.16:0/1869504725 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0106c0 con 0x7f05d406ad10 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 -- 172.21.15.16:0/1869504725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 msgr2=0x7f05d406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 --2- 172.21.15.16:0/1869504725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d406b140 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f05d406a720 tx=0x7f05cc00b4f0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 -- 172.21.15.16:0/1869504725 shutdown_connections 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 --2- 172.21.15.16:0/1869504725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d406b140 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 --2- 172.21.15.16:0/1869504725 >> v2:172.21.15.177:3300/0 conn(0x7f05d4156550 0x7f05d4069270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 -- 172.21.15.16:0/1869504725 >> 172.21.15.16:0/1869504725 conn(0x7f05d40cd480 msgr2=0x7f05d40cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 -- 172.21.15.16:0/1869504725 shutdown_connections 2023-10-06T11:43:20.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.673+0000 7f05d8915700 1 -- 172.21.15.16:0/1869504725 wait complete. 2023-10-06T11:43:20.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.674+0000 7f05d8915700 1 Processor -- start 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 -- start start 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 0x7f05d40c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d4156550 0x7f05d406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d4153a40 con 0x7f05d406ad10 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d41548a0 con 0x7f05d4156550 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d8915700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d40c54d0 con 0x7f05d40b0ea0 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38430/0 (socket says 172.21.15.16:38430) 2023-10-06T11:43:20.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.675+0000 7f05d259c700 1 -- 172.21.15.16:0/1074359183 learned_addr learned my addr 172.21.15.16:0/1074359183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d2d9d700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d4156550 0x7f05d406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d1d9b700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 0x7f05d40c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 -- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 msgr2=0x7f05d40c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 0x7f05d40c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 -- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d4156550 msgr2=0x7f05d406c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d4156550 0x7f05d406c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 -- 172.21.15.16:0/1074359183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc00b790 con 0x7f05d406ad10 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d1d9b700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 0x7f05d40c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:20.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d259c700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f05c4007fc0 tx=0x7f05c40078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4016430 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c4016590 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d4153c90 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4016930 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.676+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d4154250 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.677+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d404c380 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.677+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f05c4003020 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.678+0000 7f05ca7fc700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 0x7f05b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.678+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f05c4012070 con 0x7f05d406ad10 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.678+0000 7f05d1d9b700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 0x7f05b4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:20.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.678+0000 7f05d1d9b700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 0x7f05b4069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:20.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.682+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05c4052780 con 0x7f05d406ad10 2023-10-06T11:43:20.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.870+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05d41549e0 con 0x7f05b40670e0 2023-10-06T11:43:20.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.873+0000 7f05ca7fc700 1 -- 172.21.15.16:0/1074359183 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72252 (secure 0 0 0) 0x7f05bc007df0 con 0x7f05b40670e0 2023-10-06T11:43:20.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 msgr2=0x7f05b4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 0x7f05b4069590 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f05bc006fd0 tx=0x7f05bc008040 comp rx=0 tx=0).stop 2023-10-06T11:43:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 msgr2=0x7f05d41463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:20.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f05c4007fc0 tx=0x7f05c40078b0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 shutdown_connections 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05d406ad10 0x7f05d41463e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f05b40670e0 0x7f05b4069590 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f05d4156550 0x7f05d406c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.875+0000 7f05d8915700 1 --2- 172.21.15.16:0/1074359183 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05d40b0ea0 0x7f05d40c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.876+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 >> 172.21.15.16:0/1074359183 conn(0x7f05d40cd480 msgr2=0x7f05d40c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.876+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 shutdown_connections 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:20.876+0000 7f05d8915700 1 -- 172.21.15.16:0/1074359183 wait complete. 2023-10-06T11:43:20.878 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:43:20.889 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2023-10-06T11:43:20.079180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386716,"kb_used_data":4316,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588004,"statfs":{"total":773094113280,"available":770650116096,"internally_reserved":0,"allocated":4419584,"data_stored":2496418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002609"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.927471+0000","last_change":"2023-10-06T11:41:50.669729+0000","last_active":"2023-10-06T11:43:07.927471+0000","last_peered":"2023-10-06T11:43:07.927471+0000","last_clean":"2023-10-06T11:43:07.927471+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:43:07.927471+0000","last_undegraded":"2023-10-06T11:43:07.927471+0000","last_fullsized":"2023-10-06T11:43:07.927471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:18.660119+0000","last_change":"2023-10-06T11:43:18.660097+0000","last_active":"2023-10-06T11:43:18.660119+0000","last_peered":"2023-10-06T11:43:18.660119+0000","last_clean":"2023-10-06T11:43:18.660119+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:43:18.660119+0000","last_undegraded":"2023-10-06T11:43:18.660119+0000","last_fullsized":"2023-10-06T11:43:18.660119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:18.660009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:18.660009+0000","last_clean_scrub_stamp":"2023-10-06T11:43:18.660009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:06.754133+0000","scrub_duration":0.00176721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.925469+0000","last_change":"2023-10-06T11:41:50.459057+0000","last_active":"2023-10-06T11:43:07.925469+0000","last_peered":"2023-10-06T11:43:07.925469+0000","last_clean":"2023-10-06T11:43:07.925469+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:43:07.925469+0000","last_undegraded":"2023-10-06T11:43:07.925469+0000","last_fullsized":"2023-10-06T11:43:07.925469+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":175,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921112+0000","last_change":"2023-10-06T11:41:51.458234+0000","last_active":"2023-10-06T11:43:06.921112+0000","last_peered":"2023-10-06T11:43:06.921112+0000","last_clean":"2023-10-06T11:43:06.921112+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:43:06.921112+0000","last_undegraded":"2023-10-06T11:43:06.921112+0000","last_fullsized":"2023-10-06T11:43:06.921112+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:50.444838+0000","last_clean_scrub_stamp":"2023-10-06T11:41:50.444838+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:02.211142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921356+0000","last_change":"2023-10-06T11:41:51.253874+0000","last_active":"2023-10-06T11:43:06.921356+0000","last_peered":"2023-10-06T11:43:06.921356+0000","last_clean":"2023-10-06T11:43:06.921356+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:43:06.921356+0000","last_undegraded":"2023-10-06T11:43:06.921356+0000","last_fullsized":"2023-10-06T11:43:06.921356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:57:37.646894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.921017+0000","last_change":"2023-10-06T11:41:51.254162+0000","last_active":"2023-10-06T11:43:06.921017+0000","last_peered":"2023-10-06T11:43:06.921017+0000","last_clean":"2023-10-06T11:43:06.921017+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:43:06.921017+0000","last_undegraded":"2023-10-06T11:43:06.921017+0000","last_fullsized":"2023-10-06T11:43:06.921017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:07.170606+0000","last_change":"2023-10-06T11:41:51.254380+0000","last_active":"2023-10-06T11:43:07.170606+0000","last_peered":"2023-10-06T11:43:07.170606+0000","last_clean":"2023-10-06T11:43:07.170606+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:43:07.170606+0000","last_undegraded":"2023-10-06T11:43:07.170606+0000","last_fullsized":"2023-10-06T11:43:07.170606+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:29.111233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.923754+0000","last_change":"2023-10-06T11:41:51.254268+0000","last_active":"2023-10-06T11:43:06.923754+0000","last_peered":"2023-10-06T11:43:06.923754+0000","last_clean":"2023-10-06T11:43:06.923754+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:43:06.923754+0000","last_undegraded":"2023-10-06T11:43:06.923754+0000","last_fullsized":"2023-10-06T11:43:06.923754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:46:05.989670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":27,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-10-06T11:43:06.924777+0000","last_change":"2023-10-06T11:41:50.456047+0000","last_active":"2023-10-06T11:43:06.924777+0000","last_peered":"2023-10-06T11:43:06.924777+0000","last_clean":"2023-10-06T11:43:06.924777+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:43:06.924777+0000","last_undegraded":"2023-10-06T11:43:06.924777+0000","last_fullsized":"2023-10-06T11:43:06.924777+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:41:49.442848+0000","last_clean_scrub_stamp":"2023-10-06T11:41:49.442848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:58.579174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.85699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.443},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.70799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.27200000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.316},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.248}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.439},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.5}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.47799999999999998}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.315},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.26000000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.38}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.433}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.56000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.376}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.443}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.34399999999999997}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.502},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.46999999999999997}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.48899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.25},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.378}]},{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.39500000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.7889999999999997,"5min":6.7889999999999997,"15min":6.7889999999999997},"last":0.32800000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.379}]},{"osd":5,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41299999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.314},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.42299999999999999}]},{"osd":7,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.40400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.63400000000000001}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:43:20.893 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.893 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T11:41:50.444838+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=50, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T11:41:49.442848+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=41, tm_sec=49, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=11, tm_min=43, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T11:43:20.894 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T11:43:28.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:28.817 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:40.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T11:43:41.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.027+0000 7fd6ffb92700 1 Processor -- start 2023-10-06T11:43:41.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.027+0000 7fd6ffb92700 1 -- start start 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6ffb92700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd6f815b540 0x7fd6f80ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6ffb92700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f815b960 con 0x7fd6f815b0c0 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6ffb92700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f806e7c0 con 0x7fd6f8153e60 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6ffb92700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f806d9d0 con 0x7fd6f815b540 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fd12d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6f815b0c0 0x7fd6f806ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54198/0 (socket says 172.21.15.16:54198) 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fd12d700 1 -- 172.21.15.16:0/4098767437 learned_addr learned my addr 172.21.15.16:0/4098767437 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fe12f700 1 --2- 172.21.15.16:0/4098767437 >> v2:172.21.15.177:3300/0 conn(0x7fd6f815b540 0x7fd6f80ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851982362 0 0) 0x7fd6ec001110 con 0x7fd6f815b0c0 2023-10-06T11:43:41.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e0003570 con 0x7fd6f815b0c0 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872550729 0 0) 0x7fd6f806e7c0 con 0x7fd6f8153e60 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6ec001110 con 0x7fd6f8153e60 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049961884 0 0) 0x7fd6e0003570 con 0x7fd6f815b0c0 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fe12f700 1 -- 172.21.15.16:0/4098767437 >> v1:172.21.15.177:6789/0 conn(0x7fd6f8153e60 legacy=0x7fd6f80b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.028+0000 7fd6fe12f700 1 -- 172.21.15.16:0/4098767437 >> v1:172.21.15.16:6789/0 conn(0x7fd6f815b0c0 legacy=0x7fd6f806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6fe12f700 1 -- 172.21.15.16:0/4098767437 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f806dbb0 con 0x7fd6f815b540 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6fe12f700 1 --2- 172.21.15.16:0/4098767437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00098e0 tx=0x7fd6f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e231f0f4f0c07e54 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec001600 con 0x7fd6f815b540 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f0011420 con 0x7fd6f815b540 2023-10-06T11:43:41.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6fc92c700 1 -- 172.21.15.16:0/4098767437 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00117c0 con 0x7fd6f815b540 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6ffb92700 1 -- 172.21.15.16:0/4098767437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b540 msgr2=0x7fd6f80ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/4098767437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce530 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00098e0 tx=0x7fd6f00091a0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6ffb92700 1 -- 172.21.15.16:0/4098767437 shutdown_connections 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/4098767437 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce530 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.029+0000 7fd6ffb92700 1 -- 172.21.15.16:0/4098767437 >> 172.21.15.16:0/4098767437 conn(0x7fd6f806c410 msgr2=0x7fd6f806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.030+0000 7fd6ffb92700 1 -- 172.21.15.16:0/4098767437 shutdown_connections 2023-10-06T11:43:41.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.030+0000 7fd6ffb92700 1 -- 172.21.15.16:0/4098767437 wait complete. 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 Processor -- start 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 -- start start 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6f8153e60 0x7fd6f814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b0c0 0x7fd6f8154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80cf000 con 0x7fd6f815b540 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80cb3a0 con 0x7fd6f815b0c0 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.031+0000 7fd6ffb92700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80cb680 con 0x7fd6f8153e60 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39902/0 (socket says 172.21.15.16:39902) 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 -- 172.21.15.16:0/2750587975 learned_addr learned my addr 172.21.15.16:0/2750587975 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:41.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fd12d700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b0c0 0x7fd6f8154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fd92e700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6f8153e60 0x7fd6f814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 -- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6f8153e60 msgr2=0x7fd6f814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6f8153e60 0x7fd6f814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 -- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b0c0 msgr2=0x7fd6f8154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b0c0 0x7fd6f8154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 -- 172.21.15.16:0/2750587975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0009420 con 0x7fd6f815b540 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6fe12f700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00093f0 tx=0x7fd6f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f001cc00 con 0x7fd6f815b540 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f001cd60 con 0x7fd6f815b540 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80cd780 con 0x7fd6f815b540 2023-10-06T11:43:41.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.032+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f001a680 con 0x7fd6f815b540 2023-10-06T11:43:41.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.033+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f80cdd40 con 0x7fd6f815b540 2023-10-06T11:43:41.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.033+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80b4050 con 0x7fd6f815b540 2023-10-06T11:43:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.034+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7fd6f001a9f0 con 0x7fd6f815b540 2023-10-06T11:43:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.034+0000 7fd6f5ffb700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 0x7fd6e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.034+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7fd6f0086030 con 0x7fd6f815b540 2023-10-06T11:43:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.035+0000 7fd6fd92e700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 0x7fd6e4069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.035+0000 7fd6fd92e700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 0x7fd6e4069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80040a0 tx=0x7fd6e8007f90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.039+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6f00531c0 con 0x7fd6f815b540 2023-10-06T11:43:41.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.229+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 --> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6f804c380 con 0x7fd6e40670e0 2023-10-06T11:43:41.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.232+0000 7fd6f5ffb700 1 -- 172.21.15.16:0/2750587975 <== mgr.4112 v2:172.21.15.177:6832/90263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72511 (secure 0 0 0) 0x7fd6f804c380 con 0x7fd6e40670e0 2023-10-06T11:43:41.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 msgr2=0x7fd6e4069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 0x7fd6e4069590 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80040a0 tx=0x7fd6e8007f90 comp rx=0 tx=0).stop 2023-10-06T11:43:41.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 msgr2=0x7fd6f80ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00093f0 tx=0x7fd6f0016000 comp rx=0 tx=0).stop 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 shutdown_connections 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6f815b540 0x7fd6f80ce260 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7fd6e40670e0 0x7fd6e4069590 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd6f8153e60 0x7fd6f814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 --2- 172.21.15.16:0/2750587975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd6f815b0c0 0x7fd6f8154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 >> 172.21.15.16:0/2750587975 conn(0x7fd6f806c410 msgr2=0x7fd6f8159640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 shutdown_connections 2023-10-06T11:43:41.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.234+0000 7fd6ffb92700 1 -- 172.21.15.16:0/2750587975 wait complete. 2023-10-06T11:43:41.238 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-10-06T11:43:41.248 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-10-06T11:43:40.083533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386716,"kb_used_data":4316,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752588004,"statfs":{"total":773094113280,"available":770650116096,"internally_reserved":0,"allocated":4419584,"data_stored":2496418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002727"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:23.414112+0000","last_change":"2023-10-06T11:43:23.414085+0000","last_active":"2023-10-06T11:43:23.414112+0000","last_peered":"2023-10-06T11:43:23.414112+0000","last_clean":"2023-10-06T11:43:23.414112+0000","last_became_active":"2023-10-06T11:41:50.669478+0000","last_became_peered":"2023-10-06T11:41:50.669478+0000","last_unstale":"2023-10-06T11:43:23.414112+0000","last_undegraded":"2023-10-06T11:43:23.414112+0000","last_fullsized":"2023-10-06T11:43:23.414112+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:23.413968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:23.413968+0000","last_clean_scrub_stamp":"2023-10-06T11:43:23.413968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:40.950254+0000","scrub_duration":0.0018392599999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:18.660119+0000","last_change":"2023-10-06T11:43:18.660097+0000","last_active":"2023-10-06T11:43:18.660119+0000","last_peered":"2023-10-06T11:43:18.660119+0000","last_clean":"2023-10-06T11:43:18.660119+0000","last_became_active":"2023-10-06T11:41:50.457934+0000","last_became_peered":"2023-10-06T11:41:50.457934+0000","last_unstale":"2023-10-06T11:43:18.660119+0000","last_undegraded":"2023-10-06T11:43:18.660119+0000","last_fullsized":"2023-10-06T11:43:18.660119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:18.660009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:18.660009+0000","last_clean_scrub_stamp":"2023-10-06T11:43:18.660009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:06.754133+0000","scrub_duration":0.00176721,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:19.344338+0000","last_change":"2023-10-06T11:43:19.344299+0000","last_active":"2023-10-06T11:43:19.344338+0000","last_peered":"2023-10-06T11:43:19.344338+0000","last_clean":"2023-10-06T11:43:19.344338+0000","last_became_active":"2023-10-06T11:41:50.458760+0000","last_became_peered":"2023-10-06T11:41:50.458760+0000","last_unstale":"2023-10-06T11:43:19.344338+0000","last_undegraded":"2023-10-06T11:43:19.344338+0000","last_fullsized":"2023-10-06T11:43:19.344338+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:19.344166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:19.344166+0000","last_clean_scrub_stamp":"2023-10-06T11:43:19.344166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:06.174306+0000","scrub_duration":0.0016985220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"35'119","reported_seq":188,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:28.389737+0000","last_change":"2023-10-06T11:43:28.389688+0000","last_active":"2023-10-06T11:43:28.389737+0000","last_peered":"2023-10-06T11:43:28.389737+0000","last_clean":"2023-10-06T11:43:28.389737+0000","last_became_active":"2023-10-06T11:41:51.457510+0000","last_became_peered":"2023-10-06T11:41:51.457510+0000","last_unstale":"2023-10-06T11:43:28.389737+0000","last_undegraded":"2023-10-06T11:43:28.389737+0000","last_fullsized":"2023-10-06T11:43:28.389737+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'119","last_scrub_stamp":"2023-10-06T11:43:28.389573+0000","last_deep_scrub":"35'119","last_deep_scrub_stamp":"2023-10-06T11:43:28.389573+0000","last_clean_scrub_stamp":"2023-10-06T11:43:28.389573+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:47:45.925860+0000","scrub_duration":0.0048805259999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:21.388050+0000","last_change":"2023-10-06T11:43:21.388013+0000","last_active":"2023-10-06T11:43:21.388050+0000","last_peered":"2023-10-06T11:43:21.388050+0000","last_clean":"2023-10-06T11:43:21.388050+0000","last_became_active":"2023-10-06T11:41:51.253591+0000","last_became_peered":"2023-10-06T11:41:51.253591+0000","last_unstale":"2023-10-06T11:43:21.388050+0000","last_undegraded":"2023-10-06T11:43:21.388050+0000","last_fullsized":"2023-10-06T11:43:21.388050+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:21.387908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:21.387908+0000","last_clean_scrub_stamp":"2023-10-06T11:43:21.387908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T17:25:09.482032+0000","scrub_duration":0.0015696289999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:22.379690+0000","last_change":"2023-10-06T11:43:22.379661+0000","last_active":"2023-10-06T11:43:22.379690+0000","last_peered":"2023-10-06T11:43:22.379690+0000","last_clean":"2023-10-06T11:43:22.379690+0000","last_became_active":"2023-10-06T11:41:51.253876+0000","last_became_peered":"2023-10-06T11:41:51.253876+0000","last_unstale":"2023-10-06T11:43:22.379690+0000","last_undegraded":"2023-10-06T11:43:22.379690+0000","last_fullsized":"2023-10-06T11:43:22.379690+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:22.379545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:22.379545+0000","last_clean_scrub_stamp":"2023-10-06T11:43:22.379545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:25.046211+0000","scrub_duration":0.0022274399999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:21.360663+0000","last_change":"2023-10-06T11:43:21.360636+0000","last_active":"2023-10-06T11:43:21.360663+0000","last_peered":"2023-10-06T11:43:21.360663+0000","last_clean":"2023-10-06T11:43:21.360663+0000","last_became_active":"2023-10-06T11:41:51.254056+0000","last_became_peered":"2023-10-06T11:41:51.254056+0000","last_unstale":"2023-10-06T11:43:21.360663+0000","last_undegraded":"2023-10-06T11:43:21.360663+0000","last_fullsized":"2023-10-06T11:43:21.360663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:21.360515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:21.360515+0000","last_clean_scrub_stamp":"2023-10-06T11:43:21.360515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T22:14:24.027181+0000","scrub_duration":0.0017790239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:22.016209+0000","last_change":"2023-10-06T11:43:22.016161+0000","last_active":"2023-10-06T11:43:22.016209+0000","last_peered":"2023-10-06T11:43:22.016209+0000","last_clean":"2023-10-06T11:43:22.016209+0000","last_became_active":"2023-10-06T11:41:51.253906+0000","last_became_peered":"2023-10-06T11:41:51.253906+0000","last_unstale":"2023-10-06T11:43:22.016209+0000","last_undegraded":"2023-10-06T11:43:22.016209+0000","last_fullsized":"2023-10-06T11:43:22.016209+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:22.016052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:22.016052+0000","last_clean_scrub_stamp":"2023-10-06T11:43:22.016052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T20:06:20.530288+0000","scrub_duration":0.0020173069999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":38,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-10-06T11:43:24.007817+0000","last_change":"2023-10-06T11:43:24.007789+0000","last_active":"2023-10-06T11:43:24.007817+0000","last_peered":"2023-10-06T11:43:24.007817+0000","last_clean":"2023-10-06T11:43:24.007817+0000","last_became_active":"2023-10-06T11:41:50.455797+0000","last_became_peered":"2023-10-06T11:41:50.455797+0000","last_unstale":"2023-10-06T11:43:24.007817+0000","last_undegraded":"2023-10-06T11:43:24.007817+0000","last_fullsized":"2023-10-06T11:43:24.007817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T11:43:24.007644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T11:43:24.007644+0000","last_clean_scrub_stamp":"2023-10-06T11:43:24.007644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:21:10.837784+0000","scrub_duration":0.001792587,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298264,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073576,"statfs":{"total":96636764160,"available":96331341824,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.443}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47099999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.434},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.60099999999999998}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.316}]},{"osd":5,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.33600000000000002}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":9.4649999999999999,"5min":9.4649999999999999,"15min":9.4649999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":8.8350000000000009,"5min":8.8350000000000009,"15min":8.8350000000000009},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":9.0229999999999997,"5min":9.0229999999999997,"15min":9.0229999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":9.5190000000000001,"5min":9.5190000000000001,"15min":9.5190000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":8.9600000000000009,"5min":8.9600000000000009,"15min":8.9600000000000009},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":4.5700000000000003,"5min":4.5700000000000003,"15min":4.5700000000000003},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.022,"15min":1.022},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":9.6189999999999998,"5min":9.6189999999999998,"15min":9.6189999999999998},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.503}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302360,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069480,"statfs":{"total":96636764160,"available":96327147520,"internally_reserved":0,"allocated":999424,"data_stored":755195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.28000000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.35299999999999998}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48499999999999999}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.433},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.502}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.55500000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":15.707000000000001,"5min":15.707000000000001,"15min":15.707000000000001},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":15.678000000000001,"5min":15.678000000000001,"15min":15.678000000000001},"last":0.497}]},{"osd":3,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":12.509,"5min":12.509,"15min":12.509},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":15.613,"5min":15.613,"15min":15.613},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Oct 6 11:42:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":15.554,"5min":15.554,"15min":15.554},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":163849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.752},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.92100000000000004}]},{"osd":7,"last update":"Fri Oct 6 11:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.89100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":6.6849999999999996,"5min":6.6849999999999996,"15min":6.6849999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":5.7880000000000003,"5min":5.7880000000000003,"15min":5.7880000000000003},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":6.8239999999999998,"5min":6.8239999999999998,"15min":6.8239999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":6.7889999999999997,"5min":6.7889999999999997,"15min":6.7889999999999997},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":6.9390000000000001,"5min":6.9390000000000001,"15min":6.9390000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45800000000000002}]},{"osd":7,"last update":"Fri Oct 6 11:42:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.51000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297684,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074156,"statfs":{"total":96636764160,"available":96331935744,"internally_reserved":0,"allocated":405504,"data_stored":164827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":4.5019999999999998,"5min":4.5019999999999998,"15min":4.5019999999999998},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.434},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48999999999999999}]},{"osd":6,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Oct 6 11:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T11:43:41.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T11:43:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.368+0000 7f7b1e9a2700 1 Processor -- start 2023-10-06T11:43:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.368+0000 7f7b1e9a2700 1 -- start start 2023-10-06T11:43:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b1e9a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b18164570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b1e9a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1806bee0 con 0x7f7b181570d0 2023-10-06T11:43:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b1e9a2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1806a7f0 con 0x7f7b181574a0 2023-10-06T11:43:41.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b1e9a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1815a130 con 0x7f7b1806cb20 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b17fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7b181570d0 0x7f7b180b2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54202/0 (socket says 172.21.15.16:54202) 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.369+0000 7f7b17fff700 1 -- 172.21.15.16:0/3418420275 learned_addr learned my addr 172.21.15.16:0/3418420275 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176218763 0 0) 0x7f7b1806bee0 con 0x7f7b181570d0 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b1cf3f700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b18164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b00003570 con 0x7f7b181570d0 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 435190971 0 0) 0x7f7b00003570 con 0x7f7b181570d0 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 msgr2=0x7f7b18164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b18164570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 >> v1:172.21.15.177:6789/0 conn(0x7f7b181574a0 legacy=0x7f7b1806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1815a410 con 0x7f7b181570d0 2023-10-06T11:43:41.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f7b08001470 con 0x7f7b181570d0 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 >> v1:172.21.15.16:6789/0 conn(0x7f7b181570d0 legacy=0x7f7b180b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b00003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b00004810 0x7f7b00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1815a130 con 0x7f7b00003a90 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0000bb10 con 0x7f7b00004810 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0000bcd0 con 0x7f7b1806cb20 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b00003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b177fe700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b00004810 0x7f7b00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b1cf3f700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 -- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 msgr2=0x7f7b000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1806cb20 0x7f7b000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 -- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b00004810 msgr2=0x7f7b00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b00004810 0x7f7b00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.370+0000 7f7b17fff700 1 -- 172.21.15.16:0/3418420275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00003570 con 0x7f7b00003a90 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b17fff700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b00003e60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c3c0 tx=0x7f7b08001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbde1a09b58a16e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800cc90 con 0x7f7b00003a90 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800ce10 con 0x7f7b00003a90 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b16ffd700 1 -- 172.21.15.16:0/3418420275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08005b70 con 0x7f7b00003a90 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/3418420275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 msgr2=0x7f7b00003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/3418420275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b00003e60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c3c0 tx=0x7f7b08001e70 comp rx=0 tx=0).stop 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1cf3f700 1 -- 172.21.15.16:0/3418420275 reap_dead start 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/3418420275 shutdown_connections 2023-10-06T11:43:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/3418420275 >> 172.21.15.16:0/3418420275 conn(0x7f7b18068030 msgr2=0x7f7b1806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/3418420275 shutdown_connections 2023-10-06T11:43:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.371+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/3418420275 wait complete. 2023-10-06T11:43:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.372+0000 7f7b1e9a2700 1 Processor -- start 2023-10-06T11:43:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 -- start start 2023-10-06T11:43:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 0x7f7b1816d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 0x7f7b18153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1816db20 con 0x7f7b00003a90 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1816a7a0 con 0x7f7b18153aa0 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.373+0000 7f7b1e9a2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1816a2f0 con 0x7f7b1814f190 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39914/0 (socket says 172.21.15.16:39914) 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 -- 172.21.15.16:0/573351938 learned_addr learned my addr 172.21.15.16:0/573351938 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b177fe700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 0x7f7b1816d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b1cf3f700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 0x7f7b18153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 -- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 msgr2=0x7f7b1816d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 0x7f7b1816d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 -- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 msgr2=0x7f7b18153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 0x7f7b18153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 -- 172.21.15.16:0/573351938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18154430 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b177fe700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 0x7f7b1816d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b1cf3f700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 0x7f7b18153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b17fff700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c9b0 tx=0x7f7b08015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800cc90 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b080072e0 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b181696e0 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08007680 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.374+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b18169ca0 con 0x7f7b00003a90 2023-10-06T11:43:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.375+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b18156ce0 con 0x7f7b00003a90 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.376+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f7b08019030 con 0x7f7b00003a90 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.376+0000 7f7afffff700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 0x7f7b04069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.376+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f7b080861d0 con 0x7f7b00003a90 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.376+0000 7f7b177fe700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 0x7f7b04069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.377+0000 7f7b177fe700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 0x7f7b04069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007ba0 tx=0x7f7b0c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.381+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b08053360 con 0x7f7b00003a90 2023-10-06T11:43:41.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.572+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f7b1804c380 con 0x7f7b00003a90 2023-10-06T11:43:41.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.576+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7b08053360 con 0x7f7b00003a90 2023-10-06T11:43:41.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.576+0000 7f7afffff700 1 -- 172.21.15.16:0/573351938 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f7b08056980 con 0x7f7b00003a90 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 msgr2=0x7f7b04069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 0x7f7b04069590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007ba0 tx=0x7f7b0c009040 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 msgr2=0x7f7b18163610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c9b0 tx=0x7f7b08015000 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 shutdown_connections 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b00003a90 0x7f7b18163610 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f7b040670e0 0x7f7b04069590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b18153aa0 0x7f7b18153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 --2- 172.21.15.16:0/573351938 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7b1814f190 0x7f7b1816d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 >> 172.21.15.16:0/573351938 conn(0x7f7b18068030 msgr2=0x7f7b18155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.578+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 shutdown_connections 2023-10-06T11:43:41.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.579+0000 7f7b1e9a2700 1 -- 172.21.15.16:0/573351938 wait complete. 2023-10-06T11:43:41.592 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T11:43:41.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T11:43:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.710+0000 7f1741d9c700 1 Processor -- start 2023-10-06T11:43:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.710+0000 7f1741d9c700 1 -- start start 2023-10-06T11:43:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.710+0000 7f1741d9c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f173c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.710+0000 7f1741d9c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c0cbf20 con 0x7f173c1595b0 2023-10-06T11:43:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.710+0000 7f1741d9c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c0cc200 con 0x7f173c159980 2023-10-06T11:43:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f1741d9c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0cc4e0 con 0x7f173c0cd850 2023-10-06T11:43:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f173c1595b0 0x7f173c0cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54210/0 (socket says 172.21.15.16:54210) 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173b7fe700 1 -- 172.21.15.16:0/1244459985 learned_addr learned my addr 172.21.15.16:0/1244459985 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980186876 0 0) 0x7f173c0cbf20 con 0x7f173c1595b0 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1728003570 con 0x7f173c1595b0 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f1732ffd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f173c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 254677609 0 0) 0x7f1728003570 con 0x7f173c1595b0 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 msgr2=0x7f173c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f173c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 >> v1:172.21.15.177:6789/0 conn(0x7f173c159980 legacy=0x7f173c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0cc7c0 con 0x7f173c1595b0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3351515812 0 0) 0x7f172c001470 con 0x7f173c1595b0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 >> v1:172.21.15.16:6789/0 conn(0x7f173c1595b0 legacy=0x7f173c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1728003a90 0x7f1728003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f1728004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1728004da0 0x7f17280051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0cc4e0 con 0x7f1728004da0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172800bc50 con 0x7f1728003a90 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172800be10 con 0x7f173c0cd850 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173bfff700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1728004da0 0x7f17280051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f173b7fe700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1728003a90 0x7f1728003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.711+0000 7f1732ffd700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f1728004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 -- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 msgr2=0x7f1728004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c0cd850 0x7f1728004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 -- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1728003a90 msgr2=0x7f1728003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1728003a90 0x7f1728003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 -- 172.21.15.16:0/1244459985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728003570 con 0x7f1728004da0 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1728004da0 0x7f17280051f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1734001fc0 tx=0x7f1734001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75351da60f884474 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734001980 con 0x7f1728004da0 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f173400c8b0 con 0x7f1728004da0 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173affd700 1 -- 172.21.15.16:0/1244459985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173400cc90 con 0x7f1728004da0 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f1741d9c700 1 -- 172.21.15.16:0/1244459985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1728004da0 msgr2=0x7f17280051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f1741d9c700 1 --2- 172.21.15.16:0/1244459985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1728004da0 0x7f17280051f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1734001fc0 tx=0x7f1734001670 comp rx=0 tx=0).stop 2023-10-06T11:43:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.712+0000 7f173bfff700 1 -- 172.21.15.16:0/1244459985 reap_dead start 2023-10-06T11:43:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.713+0000 7f1741d9c700 1 -- 172.21.15.16:0/1244459985 shutdown_connections 2023-10-06T11:43:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.713+0000 7f1741d9c700 1 -- 172.21.15.16:0/1244459985 >> 172.21.15.16:0/1244459985 conn(0x7f173c099b40 msgr2=0x7f173c087fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.713+0000 7f1741d9c700 1 -- 172.21.15.16:0/1244459985 shutdown_connections 2023-10-06T11:43:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.713+0000 7f1741d9c700 1 -- 172.21.15.16:0/1244459985 wait complete. 2023-10-06T11:43:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 Processor -- start 2023-10-06T11:43:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 -- start start 2023-10-06T11:43:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c14d0e0 0x7f173c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c0c9b60 0x7f173c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.714+0000 7f1741d9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0c72b0 con 0x7f173c0c69a0 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1741d9c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ca070 con 0x7f173c0c9b60 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1741d9c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ca350 con 0x7f173c14d0e0 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39932/0 (socket says 172.21.15.16:39932) 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 -- 172.21.15.16:0/3035555275 learned_addr learned my addr 172.21.15.16:0/3035555275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 -- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c14d0e0 msgr2=0x7f173c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1732ffd700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c14d0e0 0x7f173c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c14d0e0 0x7f173c0c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 -- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c0c9b60 msgr2=0x7f173c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c0c9b60 0x7f173c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 -- 172.21.15.16:0/3035555275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0ca630 con 0x7f173c0c69a0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f173b7fe700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f172c007390 tx=0x7f172c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c018030 con 0x7f173c0c69a0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f172c00f870 con 0x7f173c0c69a0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c0ca910 con 0x7f173c0c69a0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00fc90 con 0x7f173c0c69a0 2023-10-06T11:43:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.715+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c0b6380 con 0x7f173c0c69a0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.716+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c0b41f0 con 0x7f173c0c69a0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.717+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 84893+0+0 (secure 0 0 0) 0x7f172c01b070 con 0x7f173c0c69a0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.717+0000 7f1733fff700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 0x7f171c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.717+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f172c019070 con 0x7f173c0c69a0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.717+0000 7f1732ffd700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 0x7f171c069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.718+0000 7f1732ffd700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 0x7f171c069590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f173c14d4b0 tx=0x7f1724008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T11:43:41.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.721+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f172c0517e0 con 0x7f173c0c69a0 2023-10-06T11:43:41.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.963+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f173c04c380 con 0x7f173c0c69a0 2023-10-06T11:43:41.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.963+0000 7f1733fff700 1 -- 172.21.15.16:0/3035555275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f172c001430 con 0x7f173c0c69a0 2023-10-06T11:43:41.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-10-06T11:43:41.965 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6836/311536456","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":311536456},{"type":"v1","addr":"172.21.15.177:6836","nonce":311536456}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/990391461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6835","nonce":990391461},{"type":"v1","addr":"172.21.15.177:6837","nonce":990391461}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4974,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/501213551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":501213551},{"type":"v1","addr":"172.21.15.16:6834","nonce":501213551}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5373,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1345979178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1345979178},{"type":"v1","addr":"172.21.15.16:6835","nonce":1345979178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-10-06T11:43:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 msgr2=0x7f171c069590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 0x7f171c069590 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f173c14d4b0 tx=0x7f1724008040 comp rx=0 tx=0).stop 2023-10-06T11:43:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 msgr2=0x7f173c0c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f172c007390 tx=0x7f172c001e50 comp rx=0 tx=0).stop 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 shutdown_connections 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173c0c69a0 0x7f173c0c6d70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:6832/90263,v1:172.21.15.177:6833/90263] conn(0x7f171c0670e0 0x7f171c069590 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f173c0c9b60 0x7f173c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 --2- 172.21.15.16:0/3035555275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f173c14d0e0 0x7f173c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 >> 172.21.15.16:0/3035555275 conn(0x7f173c099b40 msgr2=0x7f173c06e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 shutdown_connections 2023-10-06T11:43:41.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-10-06T11:43:41.965+0000 7f1741d9c700 1 -- 172.21.15.16:0/3035555275 wait complete. 2023-10-06T11:43:41.969 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2023-10-06T11:43:41.979 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T11:43:41.979 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T11:43:41.980 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T11:43:41.980 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:41.988 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T11:43:41.988 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T11:43:41.988 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.037 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T11:43:42.037 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T11:43:42.038 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.048 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T11:43:42.048 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T11:43:42.049 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.100 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T11:43:42.101 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T11:43:42.101 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T11:43:42.101 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.126 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T11:43:42.126 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T11:43:42.126 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.198 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T11:43:42.198 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T11:43:42.198 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.261 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T11:43:42.261 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T11:43:42.261 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.332 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T11:43:42.332 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T11:43:42.332 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.359 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T11:43:42.360 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T11:43:42.360 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.429 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T11:43:42.430 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T11:43:42.430 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.510 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T11:43:42.510 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T11:43:42.510 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.584 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T11:43:42.584 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T11:43:42.585 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T11:43:42.585 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.607 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T11:43:42.607 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T11:43:42.607 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.651 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T11:43:42.651 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T11:43:42.652 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T11:43:42.652 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.662 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T11:43:42.662 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T11:43:42.662 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.702 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T11:43:42.702 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T11:43:42.702 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T11:43:42.753 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T11:43:42.753 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T11:43:42.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T11:43:42.784 INFO:teuthology.orchestra.run.smithi016.stdout:1696592510.4444432 mon.a (mon.0) 111 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T11:43:42.784 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T11:43:42.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T11:43:42.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T11:43:42.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-06T11:43:42.996 INFO:teuthology.orchestra.run.smithi016.stdout:1696592510.4444432 mon.a (mon.0) 111 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T11:43:42.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:42.996 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T11:43:43.191 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:43.191 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T11:43:43.274 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:43.274 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T11:43:43.343 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:43.343 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T11:43:43.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:43.423 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T11:43:43.583 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:43.583 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T11:43:43.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:43.666 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T11:43:43.730 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:43.730 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T11:43:43.786 INFO:tasks.ceph:Archiving mon data... 2023-10-06T11:43:43.788 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/data/mon.a.tgz 2023-10-06T11:43:43.788 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-10-06T11:43:43.804 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.KnuAT9zccj 2023-10-06T11:43:43.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.KnuAT9zccj 2023-10-06T11:43:43.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.KnuAT9zccj 2023-10-06T11:43:44.037 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.KnuAT9zccj is 337KB 2023-10-06T11:43:44.130 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.KnuAT9zccj 2023-10-06T11:43:44.145 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/data/mon.b.tgz 2023-10-06T11:43:44.145 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T11:43:44.160 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.wglw7KHAUe 2023-10-06T11:43:44.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wglw7KHAUe 2023-10-06T11:43:44.312 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.wglw7KHAUe 2023-10-06T11:43:44.393 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.wglw7KHAUe is 459KB 2023-10-06T11:43:44.489 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.wglw7KHAUe 2023-10-06T11:43:44.503 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/data/mon.c.tgz 2023-10-06T11:43:44.503 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T11:43:44.556 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.eJUGYCCC6q 2023-10-06T11:43:44.556 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.eJUGYCCC6q 2023-10-06T11:43:44.695 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.eJUGYCCC6q 2023-10-06T11:43:44.776 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.eJUGYCCC6q is 459KB 2023-10-06T11:43:44.875 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.eJUGYCCC6q 2023-10-06T11:43:44.888 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T11:43:44.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T11:43:44.891 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T11:43:44.952 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T11:43:44.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T11:43:44.958 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T11:43:44.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T11:43:45.017 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T11:43:45.023 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T11:43:45.026 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi016/crash 2023-10-06T11:43:45.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T11:43:45.054 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi177/crash 2023-10-06T11:43:45.055 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T11:43:45.086 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T11:43:45.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:45.099 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:45.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:45.219 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T11:43:45.243 INFO:tasks.ceph:Compressing logs... 2023-10-06T11:43:45.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:43:45.262 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:43:46.085 INFO:tasks.ceph:Archiving logs... 2023-10-06T11:43:46.085 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi016/log 2023-10-06T11:43:46.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T11:43:46.367 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi177/log 2023-10-06T11:43:46.368 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T11:43:46.486 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T11:43:46.496 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T11:43:46.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T11:43:46.500 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T11:43:46.589 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T11:43:46.599 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T11:43:46.599 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:43:46.603 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T11:43:46.614 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-10-06T11:43:46.616 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-10-06T11:43:46.617 INFO:teuthology.orchestra.run.smithi016.stdout:210 Number of sources = 4 2023-10-06T11:43:46.617 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T11:43:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-10-06T11:43:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 -504us[ -509us] +/- 76ms 2023-10-06T11:43:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -388us[ -393us] +/- 53ms 2023-10-06T11:43:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +999us[ +999us] +/- 72ms 2023-10-06T11:43:46.618 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:43:46.619 INFO:teuthology.orchestra.run.smithi177.stdout:210 Number of sources = 4 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 -484us[ -484us] +/- 76ms 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 28 -385us[ -382us] +/- 53ms 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +1023us[+1023us] +/- 72ms 2023-10-06T11:43:46.620 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T11:43:46.620 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T11:43:46.645 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T11:43:46.645 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T11:43:46.656 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T11:43:46.756 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T11:43:46.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T11:43:46.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:46.878 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T11:43:46.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:46.907 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T11:43:46.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T11:43:46.928 INFO:teuthology.task.internal:Duration was 795.187315 seconds 2023-10-06T11:43:46.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T11:43:46.939 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T11:43:46.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T11:43:46.941 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T11:43:46.985 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T11:43:46.991 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T11:43:47.229 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T11:43:47.230 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2023-10-06T11:43:47.230 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T11:43:47.249 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T11:43:47.250 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T11:43:47.289 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T11:43:47.289 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:43:47.292 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T11:43:47.367 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T11:43:47.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T11:43:47.372 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T11:43:47.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T11:43:47.491 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T11:43:47.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T11:43:47.516 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T11:43:47.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T11:43:47.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:43:47.559 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:43:47.581 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2023-10-06T11:43:47.604 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2023-10-06T11:43:47.618 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:43:47.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:47.649 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T11:43:47.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T11:43:47.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T11:43:47.684 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T11:43:47.684 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi016 2023-10-06T11:43:47.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T11:43:47.727 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417/remote/smithi177 2023-10-06T11:43:47.727 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T11:43:47.763 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T11:43:47.763 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T11:43:47.766 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T11:43:47.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T11:43:47.825 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T11:43:47.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T11:43:47.837 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T11:43:47.838 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T11:43:47.840 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T11:43:47.852 INFO:teuthology.orchestra.run.smithi016.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 11:43 /home/ubuntu/cephtest 2023-10-06T11:43:47.866 INFO:teuthology.orchestra.run.smithi177.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 11:43 /home/ubuntu/cephtest 2023-10-06T11:43:47.868 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T11:43:47.880 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T11:43:47.909 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T11:43:47.933 DEBUG:teuthology.nuke:shortname: smithi016 2023-10-06T11:43:47.934 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:43:47.955 DEBUG:teuthology.nuke:shortname: smithi177 2023-10-06T11:43:47.955 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T11:43:47.959 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:22:49.413416', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBETzpEEBd5S9AGT46sVfXh0CmuWgCfE2KYq3uvkh21YMtkyX3fg8MjFIIC2e1shdrgADEeMosd/3XxXSmLp/6sY='} 2023-10-06T11:43:47.977 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:54:57-fs-wip-yuri3-testing-2023-10-05-0943-quincy-distro-default-smithi/7412417', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 11:22:49.414813', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOHDdXoRoHZfnCPeoRS4nfiBlTkz0J5rsDHjszoMelrS3lw0SWeqNkRoWAXZuJv2SwvFbAftTTP/wEmA2FuoN0='} 2023-10-06T11:43:48.013 INFO:teuthology.orchestra.console.smithi016:Power off 2023-10-06T11:43:48.013 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:43:48.035 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-10-06T11:43:48.036 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:48.050 INFO:teuthology.orchestra.console.smithi177:Power off 2023-10-06T11:43:48.050 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T11:43:48.059 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-10-06T11:43:48.072 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-10-06T11:43:48.072 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:48.095 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-10-06T11:43:52.061 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:52.096 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:52.172 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-10-06T11:43:52.207 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-10-06T11:43:56.176 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:56.208 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T11:43:56.286 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-10-06T11:43:56.317 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-10-06T11:43:56.388 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-10-06T11:43:56.418 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-10-06T11:43:56.523 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/finisher_per_module}} duration: 795.1873149871826 failure_reason: '"1696592510.4444432 mon.a (mon.0) 111 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T11:43:56.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T11:43:56.676 INFO:teuthology.run:FAIL